@dabeaz Thanks for the EuroPython talk. I really enjoyed it.
A comment for 24:31: I think the function f has a type equivalent to NoneType -> Int not Void -> Int. However in general there is no function with type NoneType -> A.
Nonetheless AFAIK the statement "you can create from a Void" is true but the usual argument goes like this:
P1: I can create anything from a Void.
P2: No, you cannot.
P1: Then give me a Void and prove me wrong.
P2: ....
The whole idea about recursive type definitions immediately made my mind jump to the problems JSON is causing the mypy people :D Although, I think the problem there is a different kind of recursion. Not as much, can you solve it with recursion, but more, how should it be implemented.
Well, as I understand it, the definition of the type of a JSON object is inherently recursive, something like JSON_TYPE = Union[int, float, str, bytes, List[JSON_TYPE], Dict[str, JSON_TYPE]], or something like that, (and of course depending on the JSON spec, I can never remember I think one of them allows for a single object at the top level and the other one does not). Anyway, the problem is that mypy doesn't support that kind of recursive definitions, leaving you with no way to create the true type definition for a JSON object. So what people usually end up doing in the meantime is to either give up on the fact that it is JSON and write out the actual type it has, or make a good-enough JSON type, which just repeats the definition 10 times or something like that, effectively giving you a 10 layers deep type definition (or course something goes wrong it is impossible to actually decipher where it goes wrong, because the string representation of that type in the error message is like a kilometer long :D )
There is a long discussion here: https://github.com/python/typing/issues/182#issuecomment-186306678 with this particular comment being the example for the good-enough def
Hi @dabeaz , Just a thankful note. 🙂I came across your books & talks via "Fluent Python" book 2 months ago and since then I am a totally changed Python programmer ..... Your teaching method is just so easily digestible. Many thanks to you for making amazing contents available. ..... I learnt more in past 2 months than what I learnt in past many years of python programming. Your lectures really helped me break the fear of diving deeply into the dirty details of python implementation.....Thank you so much. 🙏🏽🙂 Take Care.
Hi @dabeaz , I really like the explanations you give in all of your talks. That is why I'm here and would like to ask few questions because I don't know where else to look (maybe you can give me directions). I'm curious as to why when I create a pandas dataframe object and populate it with data in run-time, the python process occupies less memory (300 MB) after populating the dataframe is completed compared to when I first save the dataframe to disk via pickle then load it in another process (windows task manager informs me that it initially uses 17700 MB, then it goes down to 7200 MB after 30 min. in the mean time the process is just waiting for my input in jupyter notebook)? Does this behaviour have to do with windows or does it have to do with python? (The data I generate is about 10 GB when it is saved to disk as a file. The same thing happens with huge dictionaries, lists and numpy arrays as well). Is there a way that I can force the process to use as little memory as possible ( like 300 MB) at will when I load data in another python process?
The short answer is "yes"--meaning that it has to do with pickle, and pandas, and Windows--all at once. I don't exactly know how Pandas interacts with pickle, but it could be quite inefficient--involving a lot of data copying and larger data structures than what Pandas ordinarily uses internally.
If you want to save memory on I/O, I'd investigate different file formats (other than pickle). No specific recommendations, but it's a common problem (so there are likely multiple solutions to this).
@dabeaz recently i found myself wanting to set class properties by writing them as decorators. not something python normally does ( why isn't there a decor !? maybe because of people like me... ;P ). I've gotten inspired by your talk on sly, specifically by 'everything is possible with metaclasses' and after a lot of coffee i've made it work: https://github.com/MarcinKonowalczyk/decsetter
^ i thought you might find it interesting / sacrilegious
In sec. 6.6 of Python Distilled, there is a text "Another application of extended generators is using functions to encapsulate a “worker” task." It's better to change "extended" to "enhanced".
https://twitter.com/dabeaz/status/1454055942799171592 for anyone who needs the link
2 replies
@loneshark99 personally I would advice you to find an open source project that you enjoy and contribute something there. WTG learning programming there really is no substitute to doing it, and if not at work it will have to be spare time. Even the task of reading someone elses project (if it is not too complicated) is a really fruitful experience and then again makes it easier to contribute. Personally e.g. I've read minimalmodbus (which I needed for work) and td-watson and contributed to the latter.
tried several ones, and found a comment from David on the anyio project, asking them to drop any support for Curio (from Jan this year)
- tried several ones and started hitting issues with missing libraries
I think it'd be great if there was a websocket library for Curio. Regarding
anyio, Curio's approach to async environments has long been problematic to projects like that in some subtle ways (e.g., for example, having Event.set() be an async method). It was never a goal for Curio to be a "drop-in" replacement layer for asyncio nor am I inclined to compromise its whole fundamental design to make it work in that way. I asked for Curio support to removed from anyio mainly because I don't want it held up as a constant thorn in the side of whatever it is they're trying to do with that.
Not easy to buy Python Distilled in Europe! And almost impossible to know how many pages or the ISBN. Has this information changed recently? Many Amazon sites have conflicting information (in France, they show a 750-page book, in UK, 350, in US it used to be 250 but apparently now it's 350...). In any case, it's been removed from all European Amazon shops and their product page remains with the old title "Python Essential Reference 5th Edition". It's a complete mess :D
2 replies
Impossible to get anything out of Amazon's declining support, so I finally bought it from Book Depository, a good alternative if the delivery delay is not an issue (5-10 days). It was actually cheaper, though that was not my concern here - at least they had it in stock and with the correct reference. Looking forward to another great book!