- Gen I has them all mixed around.
- Gen II has no surprises, but I figured it's good to be thorough.
- Gen III has the first 251 in order, then a big break, then the
third-gen Pokémon mixed around, though families are usually together.
- Gen IV has the 493 in order and then alternate forms after Arceus,
which will be useful to have once Gen V comes and we have to bump
the alt forms in the pokemon table forward.
The Gen III data didn't have any errors, and I assume our Gen IV data is
much more recent and trustworthy and isn't worth checking. Crystal
tutor compatibility is stored right after HMs, so it was easy; I don't
know about any other tutors.
Gen III and IV only seem to shy-hyphenate compound words; I determined
whether or not to use a shy hyphen by looking at other instances of the
word. If it's consistently not hyphenated or just hyphenated on a line
break, I figure they mean for it to be a compound word, e.g.
"kindhearted" rather than "kind-hearted".
"Supereffective" is weird, but they seem to consistently spell it as all
one word when it's an attributive adjective, only ever hyphenating it on
a line break and only spacing it as a predicative adjective. So I
counted it as a compound word in the flavour text for Filter and Solid
Rock.
"Fire-\nand Ice-type" should be displayed "Fire- and Ice-type", but the
flavour text rendering can't tell that it's not "Fire-and". Added zero-
width spaces to invisibly separate these hyphens from the newlines,
preventing them from being interpreted as hyphenated words split over
two lines.
Items with the same name are considered the same. So, for example,
Storage Key is all one item, even though there are multiple storage keys
named "Storage Key" across the generations. As far as I know, this only
ever affects miscellaneous keys.
The Itemfinder is considered the same item as the Dowsing MCHN. They
have the same Japanese name and do the same thing; as far as I'm
concerned, the name change is just another data change.
I wrote effects for the newly-added items very quickly. They aren't
very good. I'm leaving it up to whoever takes care of issue #247 to
write good ones.
I meant to include this in the last commit. Whoops.
Rotom's description is *really long*, so I needed to bump the length up
to fit it. Also changed it to an RstTextColumn.
They now use our modified reST to link a few things like "Gracidea",
mention HG/SS where applicable, and are much more correct in general.
I might have missed some odd thing, and there are still a couple of
stylistic issues. Rotom's description is really long, for example, and
I'm not sure what to do about that; all of it seems fairly important.
This adds Japanese, French, German, Spanish, and Italian names, as
ripped from SoulSilver (Japanese) or Platinum (everything else).
This also fixes a couple of backrefs.
Gen II move flavour sometimes has shy hyphens; these, like in the
Pokémon flavour text, are represented by U+00AD SHY HYPHEN even though
the Unicode standard specifies that it be used to mark where a shy
hyphen *could* go rather than where one was placed. (Supposedly, at
least; I haven't read it for myself.)
I compared with a rip from a Mystery Dungeon game. These are the only
two that didn't match, ignoring accents on capital letters. I need to
find an official list of names that includes accents on capital
letters....
We had D/P flavour text in the abilities table already, but I didn't
entirely trust it, so I reripped it along with the rest when I moved
flavour text into its own table. And we didn't actually use the D/P
text anywhere, so I'm just going to pretend that it is entirely new.
Page breaks are represented by form feeds and soft hyphens are
represented by soft hyphens, even though the Unicode standard's idea of
a soft hyphen is different from what we mean here.
My ripping scripts are at http://github.com/Zhorken/pokemon-flavour
- Everything now accepts -i, -e, -q, and -v.
- Plumbing commands now announce what database/index they're using and
where they got them from.
- New command status, which does nothing but still does the announcing.
- New command reindex, which recreates only the whoosh index.
- encounter_type_id -> encounter_terrain_id
- Added a version_id column. Previous rates were from Diamond and
HeartGold; these have been copied to Pearl & Platinum and SoulSilver,
respectively, which i assume is accurate. RBY rates need to be added.
Based on a Platinum text dump; I'm pretty sure Conversion2 was all one
word at some point.
Interestingly, the use messages for U-turn all read "___________ used
U-Turn!", but it's "U-turn" as the actual move name.
- Wobbles are based on WHICH number is greater than some pivot, not how
many. This was making everything totally wrong, especially 0 wobbles.
- HG/SS balls all modify capture rate, rather than ball bonus.
- Everything really is integer math; even the sqrts. Bonuses are
relative to 10, not 1. HP is now treated as integer math, too.
- Implemented a minor game bug with very hard to catch Pokémon.
Now state is held within an object, rather than passed back to the
caller who must then pass it in again. That was retarded and I don't
know why I ever did it.
Code is much cleaner now.
With apologies to anyone running annotate.
Language codes are ISO 639-1; country codes are ISO 3166-1 alpha-2.
The country codes are important to keep for flags and stuff, I guess,
but reporting the language code as a short form for the language is
more correct.
Gonna see if I can do that, I guess. I added the language codes mostly
just because I was adding languages.
The only differences from Platinum are that Shuckle holds a Berry
Juice, Sky Shaymin holds a Lum Berry, and the *rizers are only held by
the final forms, only 5% of the time.
Every flavor page should work with no missing sprites. Save perhaps for
Unown, because I honestly don't have them.
Every sprite exists as ###-form.png. There is also still a ###.png,
containing a reasonable default form, so people who don't give a crap
about this mess can just use the numbered sprites. Beta forms should
now all be ###-beta.png.
Form groups now have a notion of "in-battle", which is used to hide
overworld sprites when appropriate.
Form sprites have a first-class sense of being a default or not, too.
Deoxys is... well, let's not talk about Deoxys. Deoxys is fixed.
Taken from http://www.pokepedia.fr/ (Liste des Pokémon dans l'ordre du
Pokédex National). They apparently took them from the French Mystery
Dungeon games (Poképédia:Conventions de Style).
This also corrects some typos.
This had been done before, but some of the changes were lost when I
re-ripped Diamond and Pearl.
Also, Turnback Cave has been collapsed into seven sections rather than
four. The previous change in particular ignored that the encounter
rates for the first three areas were lower than elsewhere. I'm
conjecturing wildly, but I believe those first three are the actual
pillar rooms, and the following four identical groups are the groups of
rooms between the pillars.
Conditions are now condition values; condition groups are conditions.
Types are now terrain. Slots are first-class things.
Encounters' condition values and slots' conditions have been broken off
into their own tables, as HG/SS has several slots affected by multiple
conditions.
This also fixes an absolute TON of errors with evolved Pokémon learning
a move both at level 1 and the pre-evolution's level, as well as
miscellaneous other problems.
Only the version group a forme actually exists in now has any moves for
that forme.
In addition, Deoxys formes were not showing any gen 3 moves at all
previously, because they were marked as only existing in gen 4. This
has been fixed.
Also fixed roomaji conversion to not die spectacularly when given
hiragana. For some reason I let it know about hiragana soukuon and
youon, but nothing else, so it gets totally confused.
This gives the correct ordering to level-up moves that have the same
level.
It also fixes move errors with Wartortle, Blastoise, Persian, Golduck,
Rapidash, Kabutops, Croconaw, Feraligatr, Noctowl, Sharpedo, Piplup's
family, Shinx's family, and Yanmega. Yikes.
Wrote a little add() function to clean up the duplication of
add_document().
Delete the index directory if it exists and we're being forced to
recreate it.
English fuzzy matches are preferred, followed by Roomaji and then
everything else.
The return tuple from lookup() now has a `name` parameter for the actual
name that was matched.
The setup command loads the default data into a default location, then
creates a whoosh index in a default location.
get_index is now open_index and can be made to explicitly recreate the
index. It also actually opens the index if it already existed, even
across processes, now that FileStorage is working.
The lookup command takes no switches for aiming at a different database;
it only uses the default data stores.
csvimport is now load; csvexport is now dump.
Both take an optional -e switch to specify an engine, but will happily
use a default SQLite database in the pokedex package directory.
Additionally, the CSV directory is now controlled by the optional -d
switch, and defaults to Doing The Right Thing.
So `pokedex load` now does exactly what you'd expect: loads the data
from the right files into a consistently-located database.
Good news: This no longer relies on InnoDB's default row order.
Bad news: InnoDB in MySQL 5.0 has a bug where it will sort rows
physically according to a secondary index, if there's a composite
primary key and a single-column index and the phase of the moon is
right. So a couple tables have been, once again, reordered -- but
correctly this time.
Good news: This bug will no longer fuck me up!
Whoosh's spelling module unfortunately ignores any "words" that don't
look like words, even though the algorithm words fine with arbitrary
input.
I had to clone some code from whoosh.spelling, but avoiding the
isalpha() check solved a bunch of problems. Now the index happily
compares against anything I feed into it.
Curse's type_id was 0, which is bogus; this has been fixed by creating a
real ????? type.
Fourth-gen moves all had zero as a contest effect id, which was also
bogus.
Pokémon 494 and 495 were junk and have been scrapped entirely.
pokemon_form_groups's description column was too short.
pokedex's connect() now takes kwargs passed to sessionmaker().
A more major change: some tables, like pokemon, are self-referential and
contain rows that refer to rows later in the table (for example, Pikachu
evolves from Pichu, which has a higher id). At the moment such a row is
loaded, the foreign key is thus bogus. I solved this by turning on
autocommit and wrapping add() in a try block, then attempting to readd
every failed row again after the rest of the table is finished. Slows
the import down a bit, but makes it work perfectly with foreign key
checks on.
Types, abilities, egg groups, and stats for Shaymin, Giratina, and Rotom forms.
Updated height and weight for Shaymin and Giratina forms.
Added Giratina's form descriptions and updated Shaymin's to mention link
battles and freezing.
All Solaceon Ruins rooms are identical, so there is no reason to have
them duplicated.
All the Old Chateau rooms are similarly identical, EXCEPT for the lone
room that can spawn Gengar. I also left the Rotom room in, for when I
get around to adding event encounters.
Great Marsh is now Great Marsh instead of Safari Zone.
Ruin Maniac Tunnel has been consolidated into one location, and the
areas are actually named informatively.
Turnback Cave has more appropriate area names.
Routes with two parts now mention the cardinal direction in the area
names. Also, several town names have been fixed.
Lake Verity's area names now mention WHAT they are before/after.
Apparently the secret property on a singleton hidden in the guts of
SQLAlchemy has been made private recently, so what I wanted to do (get a
list of all ORM classes) is now impossible. I gave up on trying to find
a real solution and just slapped together something using dir().
This does NOT actually change the data at all! These tables were
apparently created with no key defined, so the rows were in arbitrary
order -- but when I created and populated the tables in MySQL on
nyarumaa, the keys were defined correctly, and InnoDB ordered them by
key. This is about what should happen anyway and the discrepancy adds
clutter when dumping corrections, so I'm just committing the new order.
It used to abruptly abort if a csv file were missing, which wasn't very
nice when I'd just added a new table definition and was trying to reload
everything else.
Now it prints a status per table while loading, and will declare missing
tables to be... missing.
Finally! Location order is the same as from the old dex, which was
something like the game but ultimately arbitrary, so it's not any better
now.
This takes a very different approach to storage, rather than copying the
game exactly and trying to fix everything in code. Comments coming
shortly so other people can actually make use of this.
Tables weren't being defined as UTF-8 if that wasn't the server default.
A lot of tables were trying to create erroneous auto_increment columns.
Foreign key checks were pretty much fucking everything up.
