For questions about contributing to pandas, please ask in the pandas-dev/community channel
I'm using PySpark on databricks notebook and facing an error due to dataframe.copy() method and needed some help to understant
df_c= df.copy() File "/databricks/python/lib/python3.7/site-packages/pandas/core/generic.py", line 5811, in copy
data = self._data.copy(deep=deep)
File "/databricks/python/lib/python3.7/site-packages/pandas/core/generic.py", line 5270, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute '_data'
Hello everyone!
My name is Olga Matoula and I am a software engineer at Bloomberg London and a member of the Python community there. I recently attended a PyLadies workshop led by @MarcoGorelli guiding the participants towards their first contribution to pandas.
The week of 10-14 May Bloomberg is organizing "Guild Week", which is a week full of internal events hosted by the different communities for our Engineering department. On Friday the 7th of May there will be a Python hackathon and our idea is to have it open source focused. I want to encourage more engineers to attempt making a contribution, as well as other folks who used to contribute in sprints but have done less the past year due to cancelled conferences or other reasons. As I have found the pandas documentation very well written and easy to navigate, I have proposed the project as one of the few we will be focusing on. Other Bloomberg engineers have shared a lot of positive feedback about the project too.
It would be great if we could have the participation of a couple of the pandas maintainers during the event, to kick it off, navigate attendees through the process and suggest some issues to be worked on. It would be extremely valuable for everyone attending as problems get resolved more quickly and there's a general encouragement to have someone sharing their experience and feedback. Ideally we would need someone or some ones(!) to cover 1-2 hours each in two timezones (morning BST and EDT).
From our side, we can definitely see good participation, and even though there will be many first time contributors we can expect that good outcomes will come out of it and more advanced issues will be worked on.
Please, let me know your thoughts on this and if there are any questions. Would someone be able to help out on the day? We can always organize a chat to go through the idea in more detail and discuss further!
Olga
pandas.DataFrame.setitem . There is currently no API documentation at all for setitem, and we were wondering if there is a reason behind that? And does anyone have suggestions on fixing this issue?
np.where (as demonstrated in the third answer on the linked post, https://stackoverflow.com/a/48795837)
.locs back-to-back is already pretty efficient
DataFrame.replace is your friend. For ranges of numerical data, pd.cut might be helpful.
Hello @All
I am woking on the PDF Reading , but not able to get the perfect Package to be used, I have priorly use the pdfplumber for extracting Tables .. now this task I have to exctract teh numberic bullet points' e.g.
2. Data
2.1 New Data
2.1.1 Child data
conclusion: askjkajsjslajskjak
E_MeV_float is my float64 index
71, 72, .. with the values interpolated from the current df.
Is there a way to use a with statement when, say, extracting data from a DataFrame that is instantiated using, say, S3?
E.g.
with pd.read_csv(“s3://aws-gsod/isd-history.csv”) as data:
history = data[data[“ICAO”].astype(“str”).str.match(“KASH”)][[“WBAN”, “USAF”]].to_dict()
kash_usaf, kash_wban = history.get(“USAF”), history.get(“WBAN”)
pandas/tests/reductions/test_reductions.py::TestSeriesReductions::test_all_any_params fails, but on my local machine the test passes.
1 reply
hello
I wanted to convert a given column (containing some NaN values) into integer like this: df['x'].astype(int) and I got this error
raise ValueError("Cannot convert non-finite values (NA or inf) to integer")
ValueError: Cannot convert non-finite values (NA or inf) to integerIs there any workround for this ?
Thanks
5 replies
Hi, I'm new to open source, working on #41421. I'm trying to create the development environment but have an error on the Windows build tools path setup.
Ran:"C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvars64.bat" -vcvars_ver=14.16 10.0.17763.0
And error:
**********************************************************************
** Visual Studio 2017 Developer Command Prompt v15.8.9
** Copyright (c) 2017 Microsoft Corporation
**********************************************************************
[ERROR:vcvars.bat] Toolset directory for version '14.16' was not found.
[ERROR:VsDevCmd.bat] *** VsDevCmd.bat encountered errors. Environment may be incomplete and/or incorrect. ***How do I fix this?
4 replies
11 replies
Hi, I'm working on #41423 and would like to confirm my understanding of view vs. copy when creating a Series with copy=False.
For example, creating Series s with array r as data:
r = [1,2]
s = pd.Series(r)
s.iloc[0] = 999
print(r)
print(s)r is unchanged since s is a copy of the original data?
But when creating Series s with numpy.array as data:
r = np.array([1,2])
s = pd.Series(r)
s.iloc[0] = 999
print(r)
print(s)r is changed to [999, 2] since Series s is a view on original numpy array r?s shares the data of r.
Appreciate any help.
1 reply
4 replies
I'm only changing 1 file's comments,
pandas/core/series.py,but I have CI checks failing after I push my changes
I don't know why, or how to make them pass?
2 replies
df = df.assign(new_col=another_df['name'].values) vs df = df.assign(new_col=another_df['name'] ? It seems that when using .values there are times new_col = NaN . Can someone please explain ?
pandas.core.dtypes.base.ExtensionDtype (I also created to corresponding ExtensionArray). For now this pd.Series(my_custom_obj, dtype=MyCustomPdDtype)) works, but I would like this pd.Series(my_custom_obj)) which I find simpler, given that I know the dtype of my object.
s = pd.Series(["a", "b", "c", "a"], dtype="category")will indeed cast the values in a Categorical series, but pd.Series(["a", "b", "c", "a"]) will use object as dtype. I would like to be able to tell pandas how to behave when the passed object is of type MyCustomClass, kinda like a lookup table : ThisObject -> ThisDtype, ThatObject -> ThatDtype...
