jreback on master
REF: move mixed-dtype frame_app… (compare)
jreback on master
add match to bare pyteset.raise… (compare)
jreback on master
PERF: Using _Period for optimiz… (compare)
How does one productively add type annotations?
The contributing guide says
pandas uses mypy to statically analyze the code base and type hints. After making any change you can ensure your type hints are correct by running mypy pandas
but if I do that I get a really long list of errors
df2 = pd.DataFrame(np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]), columns=['a', 'b', 'c']) ...: In : df2 Out: a b c 0 1 2 3 1 4 5 6 2 7 8 9 In : df3 = pd.DataFrame(np.array([['D', 'A'], ['C', 'B'], [ 'X', 'Y']]), columns=['d', 'e']) In : df3 Out: d e 0 D A 1 C B 2 X Y In : df = df2.assign(**df3) In : df Out: a b c d e 0 1 2 3 D A 1 4 5 6 C B 2 7 8 9 X Y
testingmodule. If there is demand for that, we should discuss that (you can maybe open an issue for it)
My PR at pandas-dev/pandas#32817 failed at the building documentation step with the output:
build finished with problems, 2 warnings. ##[error]Process completed with exit code 1.
with the 2 warnings are
WARNING: failed to reach any of the inventories with the following issues: intersphinx inventory 'https://dateutil.readthedocs.io/en/latest/objects.inv' not fetchable due to <class 'requests.exceptions.HTTPError'>: 502 Server Error: Bad Gateway for url: https://dateutil.readthedocs.io/en/latest/objects.inv WARNING: failed to reach any of the inventories with the following issues: intersphinx inventory 'https://pandas-gbq.readthedocs.io/en/latest/objects.inv' not fetchable due to <class 'requests.exceptions.HTTPError'>: 502 Server Error: Bad Gateway for url: https://pandas-gbq.readthedocs.io/en/latest/objects.inv
Should i rerun the checks by close and reopen the PR? It looks like it's just some conectivity issue on the CI server end
Trying to use pre-commit, but I get an error from mypy:
pandas/core/groupby/generic.py:1059: error: "object" has no attribute "_data" pandas/core/groupby/generic.py:1068: error: "object" has no attribute "_data" pandas/core/groupby/generic.py:1069: error: "object" has no attribute "_data" pandas/core/groupby/generic.py:1104: error: "object" has no attribute "shape" pandas/core/groupby/generic.py:1107: error: "object" has no attribute "iloc" Found 5 errors in 1 file (checked 2 source files)
Do people get around this by just doing
$ SKIP=mypy git commit -m "descriptive commit message"
--- a/pandas/core/groupby/generic.py +++ b/pandas/core/groupby/generic.py @@ -523,7 +523,6 @@ class SeriesGroupBy(GroupBy): builtin/cythonizable functions """ ids, _, ngroup = self.grouper.group_info - result = result.reindex(self.grouper.result_index) cast = self._transform_should_cast(func_nm) out = algorithms.take_1d(result._values, ids) if cast: @@ -1455,7 +1454,6 @@ class DataFrameGroupBy(GroupBy): # for each col, reshape to to size of original frame # by take operation ids, _, ngroup = self.grouper.group_info - result = result.reindex(self.grouper.result_index) output =  for i, _ in enumerate(result.columns): res = algorithms.take_1d(result.iloc[:, i].values, ids)
no_result:Any = object()
@xhochy if I do
$ git checkout master $ git fetch upstream master $ git reset --hard upstream/master $ mypy pandas
then I get a really long list of errors, starting with
pandas/io/common.py:503: error: Argument 1 to "writestr" of "ZipFile" has incompatible type "Optional[str]"; expected "Union[str, ZipInfo]"
and ending with
pandas/__init__.py:347: error: Incompatible default for argument "dummy" (default has type "int", argument has type "__SparseArraySub") Found 201 errors in 24 files (checked 957 source files)
Do you think this means there is something wrong with my development environment? If you do
mypy pandas from the master branch do you get no errors?
Sorry to keep asking, but I still can't get around this during pre-commit.
I have run
$ pre-commit clean
I have mypy 0.730 installed:
$ version mypy 0.730
The following returns no errors:
$ mypy pandas Success: no issues found in 957 source files
but, when I try to commit the diff I showed above, I still get
mypy.....................................................................Failed - hook id: mypy - exit code: 1 pandas/core/groupby/generic.py:1064: error: "object" has no attribute "_data" pandas/core/groupby/generic.py:1073: error: "object" has no attribute "_data" pandas/core/groupby/generic.py:1074: error: "object" has no attribute "_data" pandas/core/groupby/generic.py:1109: error: "object" has no attribute "shape" pandas/core/groupby/generic.py:1112: error: "object" has no attribute "iloc" Found 5 errors in 1 file (checked 1 source file)
Might have to develop without pre-commit (at least without mypy) for now then
dmypy run -- --follow-imports=skip pandas(probably with
--timeout 3600). This would run the checks on the whole scope but through the daemon should be much faster than reloading the cache every time. I'm not sure though how large the branch switching slowdown will be.
If I'm working with an
ArrayLike (which is defined as
ArrayLike = TypeVar("ArrayLike", "ExtensionArray", np.ndarray)
) is there a way to tell mypy that it is specifically of Categorical type, i.e. that it has .categories and .codes attributes? Adding
is_categorical(if you search the mypy issue tracker there is a request to allow functions to declaring that they do narrow types, but no one has implemented)