All changes requested in PR 17f36243bc
### pokedex/main.py -
#### create_parser() -
- Change to langs argument reverting the help message to the original
one but changing the default to 'all'.
#### command_dump() -
- Add check for 'all' langs code instead of in the load.py file.
- Add/fix comments.
---
### pokedex/db/load.py -
#### dump() -
- Remove unneeded check for 'all' langs code since not it is checked in
the main.py file.
- Reword comment of what happens when the 'none' code is passed.
pokedex/main.py -
create_parser() -
Change the help message for the langs argument in the dump subparser
to show the actual default and state that the 'all' and 'none' codes
cannot be used alongside other codes.
command_dump() -
Check if 'all' or 'none' codes are passed alongside other codes. If
they are, error message is printed and program ends.
pokedex/db/load.py -
dump() -
Add check if langs code is 'all' or 'none'.
If langs wasn't passed to the parser or 'all' was passed (they are
the same since the default is 'all'), then everything will get
dumped to the csv files.
If 'none' was passed to the parser, then nothing new should be
dumped to the csv files.
pokexed/.travis.yml -
Re-added 'pokedex dump -l all' that was previously remove on
77e3d9df16Resolves: #295
When you open a file in Python 3, it defaults to using the system charset to encode the file, which is typically UTF-8 on linux systems (good) but Windows-1251 on Windows (bad).
We need to add explicit encoding=utf-8 arguments to open() calls when we open CSV files for reading or writing. To complicate matters, the csv module works only with byte strings in Python 2, and only with unicode strings in Python 3, so we can't just blindly use `io.open` everywhere.
PyPy (>= 3.6) was consistently erroring out during `pokedex load`,
with the error
OperationalError: (_sqlite3.OperationalError) cannot commit transaction - SQL statements in progress (Background on this error at: http://sqlalche.me/e/e3q8)
It turns out PyPy was not garbage collecting something, causing a
database cursor to be left open. See the giant comment block for
details.
COPY FROM FILE requires database superuser permissions,
because of the obvious security implications.
COPY FROM STDIN has no such restriction.
Also do some cleanup while we're here.
I don't really like this but ehhhhhhh, the system for prose
translations seems to be to keep them in csv/translations/, and I can't
figure out how you're supposed to DO that, plus judging by the age of
the single file that's in there, that seems to be where translations go
to die.
(Translations cannot be dumped properly because the source string hash
isn't in the database.)
By default, unofficial texts are only dumped for English, but that can
be configured if someone wants CSVs for different language(s).
Official texts (<thing>_names rows for official languages) are always
dumped.
- Everything now accepts -i, -e, -q, and -v.
- Plumbing commands now announce what database/index they're using and
where they got them from.
- New command status, which does nothing but still does the announcing.
- New command reindex, which recreates only the whoosh index.
csvimport is now load; csvexport is now dump.
Both take an optional -e switch to specify an engine, but will happily
use a default SQLite database in the pokedex package directory.
Additionally, the CSV directory is now controlled by the optional -d
switch, and defaults to Doing The Right Thing.
So `pokedex load` now does exactly what you'd expect: loads the data
from the right files into a consistently-located database.
