Commit graph

947 commits

Author SHA1 Message Date
Eevee
990469bcd5 Added foreign move names. Japanese only so far.
Also fixed roomaji conversion to not die spectacularly when given
hiragana.  For some reason I let it know about hiragana soukuon and
youon, but nothing else, so it gets totally confused.
2009-09-13 21:06:15 -07:00
Eevee
149b34b888 When loading Pokédex data, commit every 1000 rows.
Mostly done to fix RAM usage issues, but seems to have increased speed slightly.
2009-09-13 20:10:20 -07:00
Eevee
5deb9b2d16 Fixed use of split() in lookup. Now strips whitespace. #94 2009-09-13 20:09:31 -07:00
Eevee
b409a0cadf Re-ripped Diamond/Pearl level-up moves.
No errors in my old data, but some orders are different from Pt.
2009-09-12 23:15:57 -07:00
Eevee
aa8a33e45c Re-ripped Platinum moves directly.
This gives the correct ordering to level-up moves that have the same
level.

It also fixes move errors with Wartortle, Blastoise, Persian, Golduck,
Rapidash, Kabutops, Croconaw, Feraligatr, Noctowl, Sharpedo, Piplup's
family, Shinx's family, and Yanmega.  Yikes.
2009-09-12 23:02:01 -07:00
Eevee
1d4425bcb8 Raichuu, not Raichyuu 2009-09-12 14:49:02 -07:00
Eevee
e5a692b2ac Added HG/SS egg and level moves. (Thanks, Arty2) 2009-09-10 23:32:41 -07:00
Eevee
3305fce3cc Clarified some move target descriptions. 2009-09-10 10:21:09 -07:00
Eevee
774b69f5bd Made short_effect also an rST column, so effect chance works.
Added support for types in move effects.
2009-09-10 10:20:48 -07:00
Eevee
fc18219158 Added __singlename__ property to tables that have corresponding dex pages. 2009-09-10 10:17:59 -07:00
Eevee
c4b2d8fee3 Rewrote every single move effect. #55 2009-09-10 10:17:14 -07:00
Eevee
585cb31c0a Added semi-magical reST parsing for move effects. 2009-08-31 21:37:29 -07:00
Eevee
c545fbf764 Moved damage classes to a separate table. #76 2009-08-26 22:30:07 -07:00
Eevee
2853c3a42b Added alt Shaymin/Giratina/Rotom moves for Platinum. #5 2009-08-25 23:16:17 -07:00
Eevee
e2bd074146 Fix crash when stdin has no encoding. 2009-08-25 08:07:54 -07:00
Eevee
f5853c0a0b Added random lookup. #57 2009-08-24 22:59:39 -07:00
Eevee
9e3d8b317d Wildcard lookup! #89
Test suite no longer reloads the entire database.  Takes too long.

Factored out some magic numbers in lookup().
2009-08-24 22:04:55 -07:00
Eevee
11989d0db2 Some minor lookup refactoring.
Wrote a little add() function to clean up the duplication of
add_document().

Delete the index directory if it exists and we're being forced to
recreate it.
2009-08-24 21:50:19 -07:00
Eevee
25577a3397 Revert "Renamed de-accented item names in the db."
Somehow that worked..  backwards, and removed the accents.

This reverts commit 204a43b3b15946a64b971966706fba713065b7ea.
2009-08-24 21:27:21 -07:00
Eevee
69ffe01a46 Filled in missing Pokédex flavor text, even when duplicated. 2009-08-24 00:22:15 -07:00
Eevee
19152deb64 Renamed de-accented item names in the db. 2009-08-23 22:07:28 -07:00
Eevee
293bcaf724 Added whoosh and sqlite files to .gitignore. 2009-08-23 20:46:21 -07:00
Eevee
c0d6828f19 Added lookup tests. #15 2009-08-23 20:43:39 -07:00
Eevee
d00357a1ae Restored accents on Poke * items. 2009-08-23 19:18:17 -07:00
Eevee
909e61cc97 Added support for type: prefix and forme lookup. #15 2009-08-23 16:27:13 -07:00
Eevee
a7c0830e48 Return correctly-cased foreign names from lookup. #15 2009-08-22 20:49:07 -07:00
Eevee
a9a88ec3e4 Finally weight lookup results by language. #15 2009-08-22 19:44:57 -07:00
Eevee
9a20969336 Flesh out lookup by number. #15
Note that, unlike SVN veekun, entering "133" will return ANYTHING with an id of 133.
2009-08-22 18:56:02 -07:00
Eevee
2bc41e2c62 Added support for lookup by other language name. #15
English fuzzy matches are preferred, followed by Roomaji and then
everything else.

The return tuple from lookup() now has a `name` parameter for the actual
name that was matched.
2009-08-22 01:13:34 -07:00
Eevee
4e51867e95 Added lookup support for foreign language names. #15
Changed lookup()'s return value to be a list of named tuples so the
caller can know which language each result is in.
2009-08-21 00:30:01 -07:00
Eevee
0b5eba6620 open_index was returning an index instead of a SpellChecker. #15 2009-08-20 21:07:06 -07:00
Eevee
0a893be01d Allow lookup() callers to pass in their own whoosh indices. 2009-08-20 17:36:31 -07:00
Eevee
0946db6b8b Fixed some weirdness using version_group as a one-to-many. 2009-08-18 23:50:36 -07:00
Eevee
0300e01cff Removed whoosh hacks; 0.3 obviates them. #15 2009-08-18 23:50:35 -07:00
Eevee
16072ceb44 Added setup command and made lookup work sanely. #15
The setup command loads the default data into a default location, then
creates a whoosh index in a default location.

get_index is now open_index and can be made to explicitly recreate the
index.  It also actually opens the index if it already existed, even
across processes, now that FileStorage is working.

The lookup command takes no switches for aiming at a different database;
it only uses the default data stores.
2009-08-18 23:50:13 -07:00
Eevee
fd5e863eed Added --quiet switch to dump/load. 2009-08-18 18:36:45 -07:00
Eevee
1a7d046fbc Vastly improved the pokedex import/export UI.
csvimport is now load; csvexport is now dump.

Both take an optional -e switch to specify an engine, but will happily
use a default SQLite database in the pokedex package directory.

Additionally, the CSV directory is now controlled by the optional -d
switch, and defaults to Doing The Right Thing.

So `pokedex load` now does exactly what you'd expect: loads the data
from the right files into a consistently-located database.
2009-08-18 18:02:53 -07:00
Eevee
238487c908 Fixed whoosh index creation to work with 0.2.x. #15 2009-08-16 21:03:49 -07:00
Eevee
ba08a41f2f Fixed romanization of herakurosu and herugaa.
The "he" characters were erroneously hiragana instead of katakana.
2009-08-13 17:40:47 -07:00
Eevee
545cbc8928 Added generation_id to moves. 2009-08-12 17:24:35 -07:00
Eevee
a89e00acc3 Added Surfing Pikachu and Volt Tackle Pichu. #14 2009-08-09 23:08:18 -07:00
Eevee
a253df0280 Added machines table. 2009-08-09 23:07:23 -07:00
Eevee
9f9c29a398 Egg moves are passed down by the father, not the mother. 2009-08-04 16:11:53 -04:00
Eevee
ce23a82e50 Added Move.effect and VersionGroup.generation relations. 2009-08-03 22:59:39 -04:00
Eevee
e8ed55c297 Improved CSV import speed by several orders of magnitude. 2009-07-31 00:03:02 -07:00
Eevee
7566351ce1 Added growth rate formulas in LaTeX format. 2009-07-28 22:45:55 -07:00
Eevee
398545a77f Make help message readable for people without a UTF-8 terminal. 2009-07-28 18:31:06 -07:00
Eevee
6a8569b722 pokemon_moves table no longer tries to have an autoinc column. 2009-07-28 11:22:57 -07:00
Eevee
d997e27112 Changed exception syntax to work with Python 2.5. 2009-07-28 08:25:11 -07:00
Eevee
7347da558a Added Move.type relation. 2009-07-27 19:09:11 -07:00
Eevee
20d40364b1 Lookup search lowercases the term again; whoops. #15 2009-07-27 18:45:43 -07:00
Eevee
76d7bbe0e7 Added moves, types, items, and abilities to lookup index. #15 2009-07-26 23:18:11 -07:00
Eevee
bf5ce11242 Added Pokémon movesets. #14
Has an 'order' column stubbed out for ordering moves learned at the same time.
2009-07-26 23:03:10 -07:00
Eevee
64d3c7d5f1 Fixed csvexport to write in primary key order.
Good news: This no longer relies on InnoDB's default row order.

Bad news: InnoDB in MySQL 5.0 has a bug where it will sort rows
physically according to a secondary index, if there's a composite
primary key and a single-column index and the phase of the moon is
right.  So a couple tables have been, once again, reordered -- but
correctly this time.

Good news: This bug will no longer fuck me up!
2009-07-26 22:19:27 -07:00
Eevee
cce9c26125 Removed the need for a spelling column in the lookup index. #15
Whoosh's spelling module unfortunately ignores any "words" that don't
look like words, even though the algorithm words fine with arbitrary
input.

I had to clone some code from whoosh.spelling, but avoiding the
isalpha() check solved a bunch of problems.  Now the index happily
compares against anything I feed into it.
2009-07-26 00:37:37 -07:00
Eevee
bbfaf43d2b Reordered pokemon_items.csv in correct primary key order. #10 2009-07-25 20:30:22 -07:00
Eevee
1b21c953fe Added icons and database rows for HG/SS. #32 2009-07-25 17:27:27 -07:00
Eevee
25baa7c4c0 Migrated a roomaji library over from spline-pokedex.
Now includes support for small kana and youon, which were causing raw
kana output and crashes, respectively.
2009-07-25 17:21:53 -07:00
Eevee
d4077cc71d Added command_ prefix to CLI commands to fix import problems. 2009-07-25 02:43:30 -07:00
Eevee
b13ffac247 Pokédex lookup now uses a whoosh index and spell-checker. #15 2009-07-25 01:28:33 -07:00
Eevee
8fb0e550ad Stubbed in a simple lookup command. #15 2009-07-22 23:44:53 -07:00
Eevee
14a6545fa6 Fixed bad/confusing grammar in Aftermath's description. 2009-07-21 23:05:44 -07:00
Eevee
f23a3401f2 Stubbing out a lookup function. #15 2009-07-21 00:12:25 -07:00
Eevee
7c7e0484c2 Very simple items table and wild held item data. #10 2009-07-17 23:33:36 -07:00
Eevee
634ef3ed1e Fixed a slew of foriegn key import problems. #29
Curse's type_id was 0, which is bogus; this has been fixed by creating a
real ????? type.
Fourth-gen moves all had zero as a contest effect id, which was also
bogus.
Pokémon 494 and 495 were junk and have been scrapped entirely.
pokemon_form_groups's description column was too short.

pokedex's connect() now takes kwargs passed to sessionmaker().

A more major change: some tables, like pokemon, are self-referential and
contain rows that refer to rows later in the table (for example, Pikachu
evolves from Pichu, which has a higher id).  At the moment such a row is
loaded, the foreign key is thus bogus.  I solved this by turning on
autocommit and wrapping add() in a try block, then attempting to readd
every failed row again after the rest of the table is finished.  Slows
the import down a bit, but makes it work perfectly with foreign key
checks on.
2009-07-03 23:12:13 -04:00
Eevee
185264a288 Added Platinum Sinnoh dex numbers. #24 2009-06-26 21:27:35 -07:00
Eevee
4397dfbb0c Added remaining Pokémon form data. #5
Types, abilities, egg groups, and stats for Shaymin, Giratina, and Rotom forms.
Updated height and weight for Shaymin and Giratina forms.
Added Giratina's form descriptions and updated Shaymin's to mention link
battles and freezing.
2009-06-20 22:32:37 -07:00
Eevee
f9391b409a Added some Pokémon methods to hide form weirdness. #5 2009-06-15 20:48:27 -07:00
Eevee
61f973ab27 Added Platinum forms. #5 2009-06-15 20:48:02 -07:00
Eevee
e2353855be Added other languages' Pokémon names. 2009-06-07 01:36:17 -07:00
Eevee
15ee3fcccf Fixed csvimport to load in table dependency order. 2009-05-28 21:16:18 -07:00
Eevee
0af6b1c8ab Oops! Added encoding line to tables.py. #6 2009-05-17 03:01:17 -07:00
Eevee
204fe28c7a Documented crazy encounter schema. #7 2009-05-17 00:18:10 -07:00
Eevee
b9820bec29 Consolidated Solaceon Ruins and Old Chateau areas. #7
All Solaceon Ruins rooms are identical, so there is no reason to have
them duplicated.

All the Old Chateau rooms are similarly identical, EXCEPT for the lone
room that can spawn Gengar.  I also left the Rotom room in, for when I
get around to adding event encounters.
2009-05-16 23:34:24 -07:00
Eevee
29f37d89e5 Another rearrangement of CSV rows by primary key. 2009-05-16 22:31:04 -07:00
Eevee
42bcb31bdd Improved some location and area names. #7
Great Marsh is now Great Marsh instead of Safari Zone.

Ruin Maniac Tunnel has been consolidated into one location, and the
areas are actually named informatively.

Turnback Cave has more appropriate area names.

Routes with two parts now mention the cardinal direction in the area
names.  Also, several town names have been fixed.

Lake Verity's area names now mention WHAT they are before/after.
2009-05-16 22:16:19 -07:00
Eevee
8812dd9654 Fixed table loading under SQLAlchemy 0.5.3.
Apparently the secret property on a singleton hidden in the guts of
SQLAlchemy has been made private recently, so what I wanted to do (get a
list of all ORM classes) is now impossible.  I gave up on trying to find
a real solution and just slapped together something using dir().
2009-05-02 17:44:26 -07:00
Eevee
9fc09eb0ec Reordered some rows in mapping tables.
This does NOT actually change the data at all!  These tables were
apparently created with no key defined, so the rows were in arbitrary
order -- but when I created and populated the tables in MySQL on
nyarumaa, the keys were defined correctly, and InnoDB ordered them by
key.  This is about what should happen anyway and the discrepancy adds
clutter when dumping corrections, so I'm just committing the new order.
2009-05-01 06:29:19 -07:00
Eevee
d9a2d96ede Made csvimport somewhat tolerant of load errors.
It used to abruptly abort if a csv file were missing, which wasn't very
nice when I'd just added a new table definition and was trying to reload
everything else.

Now it prints a status per table while loading, and will declare missing
tables to be...  missing.
2009-05-01 06:24:09 -07:00
Eevee
9f6f210fa5 Added encounter data for Diamond, Pearl, and Platinum.
Finally!  Location order is the same as from the old dex, which was
something like the game but ultimately arbitrary, so it's not any better
now.

This takes a very different approach to storage, rather than copying the
game exactly and trying to fix everything in code.  Comments coming
shortly so other people can actually make use of this.
2009-05-01 06:20:18 -07:00
Eevee
afa6c061b6 Added a function for the EXP formula. 2009-03-27 19:49:20 -04:00
Eevee
67af605e8d Added a couple extra female back sprite flags, new in Platinum.
Renamed the female-sprite columns as for 'gen4' rather than 'dp'.
2009-03-25 20:48:36 -04:00
Eevee
85ee27dedd Fixed CSV import's handling of Boolean columns. 2009-03-25 20:43:09 -04:00
Eevee
15eae9ed6c Added Python versions of stat formulae. 2009-03-22 22:07:31 -04:00
Eevee
c4883a95f4 Added relations for evolution.
Alternate Formes also now have NULL evolution chain ids.
2009-03-20 19:28:41 -04:00
Eevee
ac325b620d CSV import now respects NULLability of columns.
Empty strings loaded into NULL columns are changed to NULL instead.
2009-03-08 21:34:48 -04:00
Eevee
20c9c23f51 Fixed some MySQL import problems.
Tables weren't being defined as UTF-8 if that wasn't the server default.

A lot of tables were trying to create erroneous auto_increment columns.

Foreign key checks were pretty much fucking everything up.
2009-03-07 18:54:01 -08:00
Eevee
774690026c Added habitat sprites. Renamed flavor to flavor_text.
pokemon_flavor_text.flavor -> pokemon_flavor_text.flavor_text
2009-03-06 19:14:42 -08:00
Eevee
e212038be0 Added Pokemon body shapes. 2009-03-04 19:29:43 -08:00
Eevee
8fa671403b Few more relations, for egg groups and evo chains. 2009-03-03 17:58:27 -08:00
Eevee
7315189b10 Added region names to the generations table. 2009-03-03 17:58:10 -08:00
Eevee
94c70f00d3 Added a generation_id column to the pokemon table. 2009-03-02 01:06:31 -08:00
Eevee
35ac91dc48 Added more table relations, for spline-pokedex Pokemon page. 2009-03-01 19:40:02 -08:00
Eevee
2e3c87b1b8 Fixed Pokemon female-sprite flags.
Pokemon that have both a female front and back sprite in Gen IV didn't
import the flags at all.  Some goof on my part.
2009-02-27 18:42:12 -08:00
Eevee
e6f5da6dab Added a whole bunch of foreign keys. 2009-02-23 23:31:35 -08:00
Eevee
77ee4eb8a0 Moved CSV sources into pokedex/.
This appears to be the only way to make data access in eggs actually
work, and is why the Platinum sprites are in pokedex/ too.
2009-02-22 23:44:40 -08:00
Eevee
bad044d1d8 Initial commit, with much of the data imported.
Includes a wrapper script 'pokedex' that can, so far, read data from a
db and spit out CSVs or deploy CSVs to a db.
2009-02-05 00:05:42 -08:00