Update README and improve docker support (#294)

* improve python 3, docker support and README

- fix support for python 3.7
- simplify docker support
- update readme
- remove obsolete item-questions file

.dockerignore

@@ -0,0 +1,5 @@
+.git
+.idea
+*.sqlite
+*.log
+site-packages

.gitignore

@@ -14,6 +14,9 @@ pip-selfcheck.json
 !/bin/edit-csv-as-yaml
 !/bin/poupdate
 !/bin/reset-postgresql-sequences
+!/bin/docker-pokedex
 /local
 /lib
 /include
+
+.idea

Dockerfile

@@ -1,23 +1,13 @@
-FROM python:2.7-slim
+ARG PYTHON_VERSION=${PYTHON_VERSION:-"3.7-slim"}
+FROM python:${PYTHON_VERSION}
 
-RUN echo "Upgrading distro..." && \
-    apt-get update > /dev/null && \
-    apt-get upgrade -y > /dev/null && \
-    echo "Installing dependencies..." && \
-    apt-get install -y git python-distribute > /dev/null && \
-    pip install --no-cache-dir virtualenv psycopg2 pymysql > /dev/null && \
-    echo "Optimizing space..." && \
-    apt-get remove --purge -y software-properties-common > /dev/null && \
-    apt-get autoremove -y > /dev/null && \
-    apt-get clean > /dev/null && \
-    apt-get autoclean > /dev/null && \
-    echo -n > /var/lib/apt/extended_states && \
-    rm -rf /var/lib/apt/lists/* && \
-    rm -rf /usr/share/man/?? && \
-    rm -rf /usr/share/man/??_*
+ADD ./ /app/
 
-COPY ./docker-entrypoint.sh /docker-entrypoint
-RUN chmod +x /docker-entrypoint
+WORKDIR /app
 
-ENTRYPOINT ["/docker-entrypoint"]
+RUN echo "Setting up project..." && \
+    pip install -e . && \
+    echo "DONE"
+
+ENTRYPOINT ["pokedex"]
 CMD ["status"]

README.md

@@ -1,31 +1,101 @@
 # pokedex
 
-This is a Python library slash pile of data containing a whole lot of data scraped from Pokémon games.  It's the primary guts of [veekun](https://veekun.com/).
+This is a Python library slash pile of data containing a whole lot of data scraped from Pokémon games.
+It's the primary guts of [veekun](https://veekun.com/).
 
 ## Current status
 
-The project is not _dead_, but it is _languishing_.  It's currently being maintained by only a single person ([eevee](https://eev.ee/)), who is also preoccupied with a lot of other things.  It needs a lot of TLC to modernize it and fix a lot of rough edges.
+The project is not _dead_, but it is _languishing_.
+It's currently being maintained by only its creator ([eevee](https://eev.ee/)) and a couple of other contributors,
+who are also occupied with a lot of other things.
 
-I started on an experiment with switching to YAML for data storage some time ago, for a variety of reasons.  It's finally starting to show some promise — all of gen 7 was dumped to a YAML format, then loaded into the database from there — but it'll take a lot more work to get this usable.  The intended _upsides_ are:
+The project needs to be modernized and to have a lot of rough edges fixed that would also make it easier
+to maintain for the games and generations to come.
 
-- The data will include everything from older games, so you don't have to guess!  Also, the site will handle older games correctly, probably!
-- Many more filtering and searching tools on veekun, since I won't have to fight SQL to write them!
-- More interesting data we've never had before, like trainer teams and overworld items!  And models?  Maps, even?  Who knows, but working on this stuff should be easier with all this existing code in place!
+### Status of the YAML migration
+
+**Update from 2017-06-18**
+
+@eevee started on an experiment with switching to YAML for data storage some time ago, for a variety of reasons.
+It's finally starting to show some promise — all of gen 7 was dumped to a YAML format, then loaded into the database
+from there — but it'll take a lot more work to get this usable. The intended _upsides_ are:
+
+- The data will include everything from older games, so you don't have to guess! 
+Also, the veekun site will handle older games correctly, probably!
+- Many more filtering and searching tools on veekun, since we won't have to fight SQL to write them!
+- More interesting data we've never had before, like trainer teams and overworld items!
+And models? Maps, even? Who knows, but working on this stuff should be easier with all this existing code in place!
 - A project that's actually documented and not confusing as hell to use!
 - A useful command line interface that doesn't require weird setup steps!
 
-If you're interested in this work, hearing about that would be some great motivation!  In the meantime, veekun will look a bit stagnant.  I can't dedicate huge amounts of time to it, either, so this may take a while, if it ever gets done at all.  Sorry.
+If you're interested in this work, hearing about that would be some great motivation! 
+In the meantime, veekun will look a bit stagnant. 
+We can't dedicate huge amounts of time to it, either, so this may take a while, if it ever gets done at all.
+Sorry.
 
 
-### How can I help?
+## How can I help?
 
-I don't know!  Not many people have the right combination of skills and interests to work on this.  I guess you could pledge to my [Patreon](https://www.patreon.com/eevee) as some gentle encouragement.  :)
+I don't know! Not many people have the right combination of skills and interests to work on this. 
+I guess you could pledge to my [Patreon](https://www.patreon.com/eevee) as some gentle encouragement. :)
 
+If you are a developer, you can of course also contribute to the development of this project via Pull Requests.
 
-## Copyright and whatnot
+### About editing CSV files
 
-The software is licensed under the MIT license.  See the `LICENSE` file for full copyright and license text.  The short version is that you can do what you like with the code, as long as you say where you got it.
+Fixing CSV data inconsistency or errors and putting that into a Pull Request it's also appreciated.
 
-This repository includes data extracted from the Pokémon series of video games.  All of it is the intellectual property of Nintendo, Creatures, inc., and GAME FREAK, inc. and is protected by various copyrights and trademarks.  The author believes that the use of this intellectual property for a fan reference is covered by fair use — the use is inherently educational, and the software would be severely impaired without the copyrighted material.
+Even though, whilst for every new game that we integrate here the initial big dump of data comes directly from
+the disassembled game files and scripts that parse their data, there is still some data that needs to be fixed or
+introduced manually.
+
+Pull Requests with manually-written modifications to the CSV files introducing data of new games without a trustful
+data source will be discarded.
+
+As mentioned, for new games we always try to automate this process with scripts that parse the disassembled game's data
+data into CSV, excepting whenever we have to fix some data inconsistency or error.
+
+Raw data can also sometimes come from external data miners, which have disassembled the game content and dumped all the
+information into human-readable text files.
+
+## Using the pokedex CLI
+
+A guide is available under the project's [Wiki](https://github.com/veekun/pokedex/wiki).
+
+### Docker support
+
+If you want to use the CLI but you don't want to install all python requirements yourself locally in your
+computer, you can use [Docker](https://www.docker.com/) and the provided Dockerfile will do everything for you.
+
+You only need to clone this project, and under the project directory, use the docker helper script to run
+any pokedex CLI command:
+
+**Examples**:
+
+Generating the SQLite database from the CSV files:
+```bash
+bin/docker-pokedex setup -v
+```
+
+Dumping the SQLite database back into the CSV files:
+```bash
+bin/docker-pokedex dump -l all
+```
+
+You also have a special command to re-build the docker image (e.g. after editing files):
+```bash
+bin/docker-pokedex rebuild
+```
+
+## License and Copyright
+
+The software is licensed under the MIT license. See the [`LICENSE`](LICENSE) file for full copyright and license text. 
+The short version is that you can do what you like with the code, as long as you say where you got it.
+
+This repository includes data extracted from the Pokémon series of video games.
+All of it is the intellectual property of Nintendo, Creatures, inc., and GAME FREAK, inc. and is protected by various 
+copyrights and trademarks. The author believes that the use of this intellectual property for a fan reference is 
+covered by fair use — the use is inherently educational, and the software would be severely impaired without the 
+copyrighted material.
 
 That said, any use of this library and its included data is **at your own legal risk**.

bin/docker-pokedex

@@ -0,0 +1,20 @@
+#!/usr/bin/env bash
+
+curr_dir="$(pwd)"
+log_file="${curr_dir}/bin/docker-pokedex.log"
+
+if [[ "${1}" = "rebuild" ]] || [[ ! -f "${log_file}" ]]; then
+    docker build --rm -t veekun/pokedex ${curr_dir}
+    echo "$(date)" >> "${log_file}"
+
+    if [[ "${1}" = "rebuild" ]]; then
+        echo "Docker container rebuilt"
+        exit
+    fi;
+fi
+
+docker run -it --rm \
+    -e TERM=xterm-256color \
+    --name "pokedex-$(date +%Y%m%d-%H%M%S)" \
+    --mount type=bind,source="${curr_dir}/pokedex/data",target=/app/pokedex/data \
+    veekun/pokedex "${@}"

docker-compose.yml

@@ -1,6 +0,0 @@
-version: '3'
-services:
-  pokedex:
-    build: ./
-    volumes:
-      - ./:/project

docker-entrypoint.sh

@@ -1,21 +0,0 @@
-#!/usr/bin/env bash
-set -e
-
-POKEDEX_DIR=${POKEDEX_DIR:-"/project"}
-
-cd ${POKEDEX_DIR}
-
-if [ ! -f "${POKEDEX_DIR}/bin/python" ] || [ ! -f "${POKEDEX_DIR}/bin/pokedex" ] ; then
-    echo "Building the bin/pokedex executable ..."
-    virtualenv $POKEDEX_DIR --python=python2
-    bin/python setup.py develop
-fi
-
-case "$1" in
-   "") bin/pokedex help
-   ;;
-   "exec") exec ${@:2}
-   ;;
-   *) bin/pokedex $@
-   ;;
-esac

docker-pokedex

@@ -1,3 +0,0 @@
-#!/usr/bin/env bash
-
-docker-compose run --rm pokedex "$@"

item-questions

@@ -1,18 +0,0 @@
-QUESTIONS ABOUT ITEMS
-- I think: key items are uncountable.  everything else is countable.  an item can be held iff it is countable.
-- how long do
-- check bp for ultimate held item ref or something
-- I need a better way to represent NPC interactions in different games
-- how much does growth mulch accelerate soil drying?  how much does damp mulch slow it?
-- does lum berry cure confusion?
-- how does sweet heart work?  it seems a guy in mistralton will trade some for heart scales?
-- kind of guessing about the * urge items
-
-CHANGES SINCE R/S
-- 4 shoal salt + 4 shoal shells, traded to a guy in Shoal Cave, make a Shell Bell
-- white flute increased encounter rate by 50%, not 100%
-- shards can be traded for the respective stones
-
-
-
-UP TO 67

setup.py

@@ -1,30 +1,29 @@
 from setuptools import setup, find_packages
 
-import sys
-
 setup(
-    name = 'Pokedex',
-    version = '0.1',
-    zip_safe = False,
-    packages = find_packages(),
-    package_data = {
+    name='Pokedex',
+    version='0.1',
+    zip_safe=False,
+    packages=find_packages(),
+    package_data={
         'pokedex': ['data/csv/*.csv']
     },
-    install_requires = [
-        'SQLAlchemy>=0.9.7',
+    install_requires=[
+        'SQLAlchemy>=1.0,<2.0',
         'whoosh>=2.5,<2.7',
         'markdown==2.4.1',
-        'construct<=2.5.3',
+        'construct==2.5.3',
         'six>=1.9.0',
     ],
-    entry_points = {
+    entry_points={
         'console_scripts': [
             'pokedex = pokedex.main:setuptools_entry',
         ],
     },
-    classifiers = [
+    classifiers=[
         "Programming Language :: Python :: 2.7",
         "Programming Language :: Python :: 3.4",
         "Programming Language :: Python :: 3.5",
-    ],
+        "Programming Language :: Python :: 3.7",
+    ]
 )