Andrew Ekstedt
c9855a2282
remove the other explicit .close() calls
2018-03-30 14:46:03 -07:00
Andrew Ekstedt
bf1044a9ce
undo the table.create(bind=engine) change
2018-03-30 14:41:25 -07:00
Andrew Ekstedt
99fa70de41
a couple more
...
also, print a message whenever we flush the row queue
2018-03-30 14:35:00 -07:00
Andrew Ekstedt
c0e588df88
explicitly close ResultProxy objects
...
session.execute() returns a ResultProxy in case you want to see the
result of your query. we don't.
2018-03-30 14:23:34 -07:00
Andrew Ekstedt
140b46875d
something something ScopedSession?
2018-03-30 14:08:10 -07:00
Andrew Ekstedt
ee1a1983c0
try only disabling PRAGMA journal_mode=OFF
2018-03-30 13:20:36 -07:00
Andrew Ekstedt
48655ce39d
try only disabling PRAGMA synchronous = OFF
2018-03-30 13:18:19 -07:00
John T. Wodder II
c3f566b2c9
Addressed all of pyflakes3's complaints
2016-11-24 21:29:58 +00:00
Eevee (Lexy Munroe)
54ea67a804
Fix pokedex dump under Python 2 AND 3, at the same time even
2015-11-04 20:10:27 -08:00
Eevee (Lexy Munroe)
e0f4d3be7a
Fix pokedex dump under Python 2
2015-11-04 20:03:08 -08:00
Eevee (Alex Munroe)
0ff24b4dc8
Fix the CLI in py3
2015-10-05 16:29:21 -07:00
Eevee (Alex Munroe)
b76b74e7a6
Compat with Python 3.3+
2015-10-05 08:12:23 -07:00
Andrew Ekstedt
93988d966c
load: Use COPY FROM STDIN on PostgreSQL.
...
COPY FROM FILE requires database superuser permissions,
because of the obvious security implications.
COPY FROM STDIN has no such restriction.
Also do some cleanup while we're here.
2015-05-30 22:48:20 -07:00
Lynn "Zhorken" Vaughan
63dac87a40
Dump translations for official languages by default.
...
I don't really like this but ehhhhhhh, the system for prose
translations seems to be to keep them in csv/translations/, and I can't
figure out how you're supposed to DO that, plus judging by the age of
the single file that's in there, that seems to be where translations go
to die.
2014-06-01 19:44:32 -04:00
Lynn "Zhorken" Vaughan
f54a4caaca
General code tidying for this Oracle fix.
2014-02-21 17:48:19 -05:00
Lynn "Zhorken" Vaughan
5b759feaa2
Sort out all the non-nullable columns with empty values.
2014-02-21 17:48:19 -05:00
Epithumia
7b2743be75
Dynamically mangle long table names for Oracle; Unicode → UnicodeText.
2014-02-21 17:47:55 -05:00
Eevee
3511bf3076
Ahem, ACTUALLY drop types when loading.
2012-03-31 21:30:05 -07:00
Eevee
987116f662
Fix duplicate type errors when loading into postgresql.
2012-03-30 23:33:48 -07:00
Petr Viktorin
41a168a0e2
Add a missing import, remove some unused ones
2011-08-30 23:05:22 +03:00
Petr Viktorin
34a8070449
Load translations in pokedex load.
2011-04-11 17:07:38 +03:00
Petr Viktorin
817c4c289d
Don't dump unofficial translations in pokedex dump
...
(Translations cannot be dumped properly because the source string hash
isn't in the database.)
By default, unofficial texts are only dumped for English, but that can
be configured if someone wants CSVs for different language(s).
Official texts (<thing>_names rows for official languages) are always
dumped.
2011-04-11 17:07:37 +03:00
Eevee
36fa8b7093
Fix a misuse of a set as a dictionary in db loading.
2011-04-10 18:51:03 -07:00
a_magical_me
bbb7cb9907
Make load.py more idiomatic.
...
Also snuck an idiom into db/__init__.py.
2011-04-03 04:26:45 -07:00
a_magical_me
9340f24ae4
load: Add --recursive option.
...
Helps somewhat with #526 (`pokedex load` is slow) by making it easier to
load only the tables you're interested in.
2011-04-03 03:17:08 -07:00
a_magical_me
b924a82236
Remove all uses of str.format().
...
For Python 2.5 compatibility.
2011-04-03 03:16:52 -07:00
Eevee
adbd3b628f
Fix pokedex load with empty tables.
2011-04-03 01:23:48 -07:00
Petr Viktorin
3bfb7f0bb9
Update the comment for the Postgres version of pokedex load.
2011-04-01 16:08:56 -07:00
Petr Viktorin
3475c86d2e
Print progress for dropping/creating tables
2011-03-29 17:42:59 +03:00
Petr Viktorin
bb4861b8c6
Faster pokedex load for PostgreSQL #526
...
Also added the -S (--safe) option, which disables the backend-specific
optimizations.
This gives over 3× speedup on my machine :)
2011-03-29 17:42:48 +03:00
Petr Viktorin
497ba412b0
Speed tweaks for pokedex load in SQLite
2011-01-27 21:51:30 -08:00
a_magical_me
febfb239fb
Python 2.5 compatibility
2010-05-25 14:41:15 -07:00
a_magical_me
ffc30bff8f
Factor out logic for finding the default db/index. #180
...
Note: `if not x:` has changed to `if x is not None:`, changing the
semantics slightly. Shouldn't be a big issue.
2010-05-13 21:45:51 -07:00
Eevee
25c81541ae
Use fnmatch for filenameish glob parsing.
2010-04-24 15:37:55 -07:00
Eevee
c9f32a8145
Loading big tables now shows a progress indicator. #180
2010-04-24 14:06:56 -07:00
Eevee
2204b95585
Overhauled CLI. #180
...
- Everything now accepts -i, -e, -q, and -v.
- Plumbing commands now announce what database/index they're using and
where they got them from.
- New command status, which does nothing but still does the announcing.
- New command reindex, which recreates only the whoosh index.
2010-04-24 14:06:56 -07:00
Eevee
c1c0225eb8
Optimized the fuck out of pokedex load.
...
Major change was to skip the ORM entirely. Sped it up 500%.
2009-12-09 18:15:51 -08:00
Eevee
149b34b888
When loading Pokédex data, commit every 1000 rows.
...
Mostly done to fix RAM usage issues, but seems to have increased speed slightly.
2009-09-13 20:10:20 -07:00
Eevee
fd5e863eed
Added --quiet switch to dump/load.
2009-08-18 18:36:45 -07:00
Eevee
1a7d046fbc
Vastly improved the pokedex import/export UI.
...
csvimport is now load; csvexport is now dump.
Both take an optional -e switch to specify an engine, but will happily
use a default SQLite database in the pokedex package directory.
Additionally, the CSV directory is now controlled by the optional -d
switch, and defaults to Doing The Right Thing.
So `pokedex load` now does exactly what you'd expect: loads the data
from the right files into a consistently-located database.
2009-08-18 18:02:53 -07:00
