When you open a file in Python 3, it defaults to using the system charset to encode the file, which is typically UTF-8 on linux systems (good) but Windows-1251 on Windows (bad).
We need to add explicit encoding=utf-8 arguments to open() calls when we open CSV files for reading or writing. To complicate matters, the csv module works only with byte strings in Python 2, and only with unicode strings in Python 3, so we can't just blindly use `io.open` everywhere.
The schema has changed a little in the past, uh, seven years.
Works well enough to parse and display one of my pokemon in the gts
plugin.
Updates veekun/spline-pokedex#72
PyPy (>= 3.6) was consistently erroring out during `pokedex load`,
with the error
OperationalError: (_sqlite3.OperationalError) cannot commit transaction - SQL statements in progress (Background on this error at: http://sqlalche.me/e/e3q8)
It turns out PyPy was not garbage collecting something, causing a
database cursor to be left open. See the giant comment block for
details.
* SQLAlchemy 1.0 introduced "baked queries" - a way to construct Query
objects so that they can be cached and reused.
* SQLAlchemy 1.2 changed lazyloaded columns to use baked queries under the
hood.
* Our MultilangQuery class attempts to set _default_language_id right
before the query is executed by overriding the __iter__ method.
* Baked queries bypass the __iter__ method and call a lower-level
method, _execute_and_instances, directly.
* This caused problems where _default_language_id wouldn't get set
correctly on lazyloaded columns.
* To fix, make MultilangQuery override the _execute_and_instances
method instead of __iter__.
* This is really just a stopgap: the root cause is that query params
are not preserved across lazyloads.
Tested with SQLAlchemy 0.9.7, 1.1.18, and 1.2.5.
Updates #236.
Made by running a script to compare an old version of moves.csv[1] with
the current version.
[1]: cb5276773b
Hidden power changed from variable power to 100.
We can't represent NULLs in the changelog,
so set the power to 1 instead.
A few effect chances changed from NULL to 100.
We can't represent this either, but i don't think it matters.
leer: effect_chance changed from NULL to 100
hidden-power: power changed from 1 to 60
chatter: effect_chance changed from NULL to 100
v-create: effect_chance changed from NULL to 100
Here's the complete list of changes:
swords-dance: pp changed from 30 to 20
whirlwind: accuracy changed from 100 to NULL
vine-whip: power changed from 35 to 45
vine-whip: pp changed from 15 to 25
pin-missile: power changed from 14 to 25
pin-missile: accuracy changed from 85 to 95
roar: accuracy changed from 100 to NULL
flamethrower: power changed from 95 to 90
hydro-pump: power changed from 120 to 110
surf: power changed from 95 to 90
ice-beam: power changed from 95 to 90
blizzard: power changed from 120 to 110
submission: pp changed from 25 to 20
growth: pp changed from 40 to 20
thunderbolt: power changed from 95 to 90
thunder: power changed from 120 to 110
minimize: pp changed from 20 to 10
barrier: pp changed from 30 to 20
lick: power changed from 20 to 30
smog: power changed from 20 to 30
fire-blast: power changed from 120 to 110
skull-bash: power changed from 100 to 130
skull-bash: pp changed from 15 to 10
glare: accuracy changed from 90 to 100
poison-gas: accuracy changed from 80 to 90
bubble: power changed from 20 to 40
psywave: accuracy changed from 80 to 100
acid-armor: pp changed from 40 to 20
crabhammer: power changed from 90 to 100
struggle: target_id changed from 10 to 8
thief: power changed from 40 to 60
thief: pp changed from 10 to 25
snore: power changed from 40 to 50
cotton-spore: target_id changed from 10 to 11
sweet-kiss: type_id changed from 1 to 18
perish-song: target_id changed from 12 to 14
charm: type_id changed from 1 to 18
fury-cutter: power changed from 20 to 40
heal-bell: target_id changed from 4 to 13
moonlight: type_id changed from 1 to 18
hidden-power: power changed from 1 to 60
future-sight: power changed from 100 to 120
heat-wave: power changed from 100 to 95
will-o-wisp: accuracy changed from 75 to 85
smellingsalt: power changed from 60 to 70
follow-me: priority changed from 3 to 2
nature-power: target_id changed from 1 to 10
knock-off: power changed from 20 to 65
poison-fang: effect_chance changed from 30 to 50
meteor-mash: power changed from 100 to 90
meteor-mash: accuracy changed from 85 to 90
aromatherapy: target_id changed from 4 to 13
air-cutter: power changed from 55 to 60
overheat: power changed from 140 to 130
rock-tomb: power changed from 50 to 60
rock-tomb: pp changed from 10 to 15
rock-tomb: accuracy changed from 80 to 95
extrasensory: pp changed from 30 to 20
muddy-water: power changed from 95 to 90
covet: pp changed from 40 to 25
wake-up-slap: power changed from 60 to 70
tailwind: pp changed from 30 to 15
assurance: power changed from 50 to 60
psycho-shift: accuracy changed from 90 to 100
aura-sphere: power changed from 90 to 80
air-slash: pp changed from 20 to 15
dragon-pulse: power changed from 90 to 85
power-gem: power changed from 70 to 80
energy-ball: power changed from 80 to 90
draco-meteor: power changed from 140 to 130
leaf-storm: power changed from 140 to 130
gunk-shot: accuracy changed from 70 to 80
chatter: power changed from 60 to 65
magma-storm: power changed from 120 to 100
wonder-room: priority changed from -7 to 0
rage-powder: priority changed from 3 to 2
magic-room: priority changed from -7 to 0
storm-throw: power changed from 40 to 60
synchronoise: power changed from 70 to 120
synchronoise: pp changed from 15 to 10
low-sweep: power changed from 60 to 65
hex: power changed from 50 to 65
incinerate: power changed from 30 to 60
water-pledge: power changed from 50 to 80
fire-pledge: power changed from 50 to 80
grass-pledge: power changed from 50 to 80
struggle-bug: power changed from 30 to 50
frost-breath: power changed from 40 to 60
sacred-sword: pp changed from 20 to 15
hurricane: power changed from 120 to 110
techno-blast: power changed from 85 to 120
Fixes#127Fixes#136
Merge pull request #162 from rubendal/master
The Cosplay Pikachu form change mechanic is similar to Rotom:
when it changes form it learns a special move, possibly forgetting
its old special move if it had already changed form.
Renamed the "Rotom Form" pokemon move method to "Form Change"
and updated the description accordingly.
Merge pull request #189 from phmatray/patch-1
The pokemon_name field in pokemon_form_names is sadly not rippable and has to be created by hand, being an unofficial amalgam of the species name and form name. Looks like Zhorken made a typo when adding the French form names originally.
Merge pull request #180 from seii/veekun-issue-103
Background: When I originally ripped FR/LG encounters, i gave them
their own set of location areas instead of reusing the ones from R/B/Y.
My reasoning was that location areas aren't really consistent across games,
and we have to name them by hand anyway, so it would be better to take care
of merging them in the presentation layer than the database layer.
The other devs disagreed with me, and all other rips have reused location areas.
This set of commits removes the separate location areas for FR/LG, merging them
with the existing location areas. Mostly straightforward, except:
- My original rip collapsed the three upper floors of the Pokémon Mansion into
a single unnamed area, since they had the same encounters. But R/B/Y had
slightly different encounters on each floor, so we have to re-duplicate these
floors.
- I'm not sure how we ended up with four Victory Roads in Kanto, but we did.
One of them (kanto-victory-road-1) is the G/S/C victory road, which has a
completely different layout so it makes sense having its own set of location
areas. As for the other three, one was used by R/B/Y (kanto-victory-road-2),
one by FireRed (kanto-victory-road-3), and one by LeafGreen
(kanto-victory-road-4). We can merge these.
Also, a few locations had inconsistent names:
- Routes 19, 20 & 21 => Sea Routes 19, 20 & 21
- Route 2 unnamed area => Route 2 south, towards Viridan City
- Safari Zone center => Safari Zone middle
Thanks to seii for doing all the hard work.
Fixes#103Fixes#156
There was a mix up with B1F and 1F in FireRed and LeafGreen.
Both floors got labeled as 1F during the location area cleanup,
and the B1F encounters were copied to 2F and 3F instead of the 1F
encounters.
This broke my database import:
```
psycopg2.DataError: extra data after last expected column
CONTEXT: COPY pokemon_shape_prose, line 21: "6,9,Upright,Sciurine,Pokémon with a bipedal, tailed form"
```
