These are chat archives for dropbox/pyston

17th
Mar 2015
Marius Wachtler
@undingen
Mar 17 2015 15:20
I'm trying to add unicode support to pypa (vinzenz/libpypa#19)
I tried now a different approach: I modified pypa to don't do anything with escape sequences inide unicode strings but instead just take the string as is and set a flag unicode and raw flag on the ast node. I then call PyUnicode_DecodeUnicodeEscape/PyUnicode_DecodeRawUnicodeEscape inside the interpreter/codegen to generate the unicode string (instead of decodeUTF8StringPtr). This works surprisingly well and is only a very small code change
Marius Wachtler
@undingen
Mar 17 2015 15:26
but feels somewhat strange that the decoding won't be part of the parsing phase / errors in escape sequences will get reported when the get executed and not during parsing
what are you guys thoughts on this? Maybe I could also try to call the cpythons functions when converting from the pypa ast to our (this would also be a performance improvement).
Chris Toshok
@toshok
Mar 17 2015 17:31
static int _IsFortranContiguous(Py_buffer* view)
I’d expect errors in escape sequences in literals to be reported at parse time, but I’m not sure how cpython does it
Kevin Modzelewski
@kmod
Mar 17 2015 17:58
hmm yeah could we do this in pypa-parser.cpp when we convert from the pypa ast to the pyston ast?
Marius Wachtler
@undingen
Mar 17 2015 18:14
I will try it out. seems like the best place.
Kevin Modzelewski
@kmod
Mar 17 2015 20:11
It worked!
Congratulations on your first Django-powered page.
Chris Toshok
@toshok
Mar 17 2015 20:11
woohoo!
Kevin Modzelewski
@kmod
Mar 17 2015 20:12
I guess the formatting didn't copy over
yeah! and this is with the sqlite database enabled
Chris Toshok
@toshok
Mar 17 2015 20:12
awesome
Marius Wachtler
@undingen
Mar 17 2015 20:18
awesome!
Marius Wachtler
@undingen
Mar 17 2015 21:08
I have added unicode literal support to pypa using a callback during parsing: undingen/pyston@e83da3e and https://github.com/vinzenz/libpypa/pull/20/files
what are you thoughts?