Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Tom Caruso
    @tomplex
    hey folks. I was wondering if there is some way to combine a Protocol with nominal typing. I want to be able to say that, for example,T is a <something>, but it also has a specific structure. Here's a gist with an example of what I mean. https://gist.github.com/tomplex/9b63c39649ddebdfb1f05991a3199d22
    6 replies
    Zhang Shihe
    @browncrane
    No matter how I change the mypy.ini config, the coverage report doesn't change a little bit. Is there any way to make it consistent with the console log? For e.g. Can I add some option like ignore_missing_imports to the config file so that some of the errors can be suppressed?
    lenards-speechmatics
    @lenards-speechmatics
    Hi, is there any way to reflect on types? Like if I have a x : TypedDict[...] then I would like to have a way to specify an y : type_of(x['foo']). Is there something like this hypothetical type_of?
    David Tucker
    @tucked:matrix.org
    [m]
    from typing import Any, List
    
    
    def foo(x: List[Any]):
        pass
    
    
    foo(x=list(range(3)) + list("abc"))
    sscce.py:8: error: Argument 1 to "list" has incompatible type "str"; expected "Iterable[int]"
    sscce.py:8: note: Following member(s) of "str" have conflicts:
    sscce.py:8: note:     Expected:
    sscce.py:8: note:         def __iter__(self) -> Iterator[int]
    sscce.py:8: note:     Got:
    sscce.py:8: note:         def __iter__(self) -> Iterator[str]
    Found 1 error in 1 file (checked 1 source file)
    How do I fix this?
    1 reply
    David Tucker
    @tucked:matrix.org
    [m]
    Gnarly haha, thanks! In my real code, the lists are actually comprehensions (in case that opens any other options). Is this behavior a bug?
    Oh, maybe I can itertools.chain 🤔
    jinsun
    @jinsun:matrix.org
    [m]

    not sure if it can be called a bug, it is technically right in that you can't add list[int] and list[str]
    OTOH the type inference could be more magical and realize that you want list[Any] in the end so mypy could allow it

    you could also cast the values to list[Any], foo(x=cast(list[Any], list(range(3))) + cast(list[Any], list("abc")))
    so that mypy doesn't try to infer the list[str] and list[int], but that is uglier

    lenards-speechmatics
    @lenards-speechmatics
    I have something like this:
    T = TypedDict[...] # defined in some external package
    
    def foo(x: T) -> ?:
         return x['foo']
    What should I write in place of ??
    David Tucker
    @tucked:matrix.org
    [m]

    it is technically right in that you can't add list[int] and list[str]

    ?

    didn't mypy used to reduce to e.g. list[object]?
    3 replies
    lenards-speechmatics
    @lenards-speechmatics

    also I have something like:

    def bar(y: ?) -> T:
        x: T = # ...
        x['foo'] = y

    with the same question

    3 replies
    Ehizojie
    @ehizojie1
    Hi there
    I have recently updated some packages in a series of python-environments (running different python versions, from py36+)
    I got the following error from the environment having pyhon3.6 as default;
    " ERROR: Package 'typing-extensions' requires a different Python: 3.6.15 not in '>=3.7' "
    Ehizojie
    @ehizojie1
    "typing-extensions 4.1.1" was previously installed. I noticed "typing-extensions 4.20" supports python3.7+.
    I want to confirm that support for python3.6 has been dropped.
    2 replies
    Michael Cochez
    @miselico
    I am getting the following error with mypy 0.950, using python 3.10, but not with 3.9.
    ... has incompatible type "Dict[str, Tuple[str, Iterable[Sequence[object]], str]]"; expected "Optional[Union[Mapping[str, Union[SupportsRead[Union[str, bytes]], str, bytes]], Mapping[str, Tuple[Optional[str], Union[SupportsRead[Union[str, bytes]], str, bytes]]], Mapping[str, Tuple[Optional[str], Union[SupportsRead[Union[str, bytes]], str, bytes], str]], Mapping[str, Tuple[Optional[str], Union[SupportsRead[Union[str, bytes]], str, bytes], str, MutableMapping[str, str]]]]]" To me this seems compatible. Is this a bug?
    1 reply
    Tory Clasen
    @tclasen
    Hello, is it possible to use the *.so files produced by mypyc for anything other than python extensions? Can I import them in a C, Rust, or Go project?
    Alex Grönholm
    @agronholm
    not really
    Zhang Shihe
    @browncrane
    Although the issue is created python/mypy#12715 I still like to try my luck here.
    The report show 44.41% imprecise, and command-line shows Success: no issues found in 1 source file
    Does that mean imprecise is not an error or report is applying the strict option implicitly?
    TDV Alinsa
    @alinsavix
    Is there any way to do a nested dict type that doesn't need to have the number of levels of nesting predefined? Basically a type definition that is pretty much like MetaValues = Union[Dict[str, 'MetaValues'], str] but without giving me a cyclic definition complaint.
    1 reply
    LEdoian
    @LEdoian:matrix.org
    [m]

    Is there some way to represent a tuple, which has, say, two ints and maybe something else following? I.e. I have code:

    BAD_LIST =  [
            (1, 2, 'I am bad'),
            (4, 5),
        ]

    Mypy tells me this is list[tuple[object, ...]], so if I went on to processing it:

    PROCESS_BAD = [(x[1], range(x[0]), ['something' for _ in range(x[1])])
        for x in BAD_LIST]

    Mypy complains about me using object as an argument to range.

    I could write all the types of the list: BAD_LIST: list[tuple[int, int, object] | tuple[int,int]] = [...], but for more complex types of data this is does not scale. (Of course, mypy could be silenced…)
    1 reply
    LEdoian
    @LEdoian:matrix.org
    [m]
    I kinda wish there was a way to annotate "some tuples that begin with numbers and then anything may follow", but I think it is currently not possible.
    And since my original problem was some (rather ad-hoc) data loading, I hoped that mypy can just understand that without hints. So I ended up silencing it, but I was curious whether there was a simpler way.
    Huon Wilson
    @huonw

    Hey all, I have a code base that uses mypy and it's great. We use a bunch of enum.Enums too, and they're also great.
    We have a bunch of places that need to map each member of an enum to some other value (e.g. a descriptive bit of text). Some approaches:

    1. Attribute on the enum MyEnum.X.fact: this doesn't scale (multiple mappings leads to a huge enum definition), doesn't work for enums in external packages, and leads to poor code architecture (e.g. putting values for human facing display into a "plain old data" enum's definition).
    2. Dictionary facts: dict[MyEnum, str] = {MyEnum.X: "foo"}: it's easy for that to get out of sync, where a member is added to the enum but not the dictionaries (e.g. maybe there's a MyEnum.Y too). (Dynamic check or test for exhaustiveness can work, but isn't quite as nice.)
    3. Function def fact(val: MyEnum) -> str containing if val is MyEnum.X: return "foo" elif val is MyEnum.Y: ... with a else: _: NoReturn = val 'type assert' at the end: really verbose.

    Is there a way to make 2 work better, by annotating it to be total (has an element for every key)? That is, type hint for a dict with enum or Literal keys that is total?
    This is something like TypeScript's Record, or generalising TypedDict to use str keys. (I've also asked this last week at https://stackoverflow.com/q/72022403/1256624 with additional explanation/code, and internet points for the best answer :) )

    lenards-speechmatics
    @lenards-speechmatics
    Is there any way to make a protocol like this work with TypedDict as I expect? https://mypy-play.net/?mypy=latest&python=3.10&gist=bb4f4cbc03a8ccd1aac5c0a1c5e1d83b
    lenards-speechmatics
    @lenards-speechmatics
    or as a more high level question, is it in any way possible to implement the following function as a generic with correct type annotations?
    def f(x: ?) -> ?:
        return x['foo']
    If I pass a TypedDict, I would like the corresponding type to the field 'foo' to be the deduced return type, not object.
    lenards-speechmatics
    @lenards-speechmatics
    There is something fishy with covariance/contravariance there, right?
    lenards-speechmatics
    @lenards-speechmatics

    This program typechecks:

    from typing import TypedDict
    
    class A(TypedDict, total=False):
        foo: str
    
    class B(TypedDict, total=False):
        foo: str
        bar: str
    
    def f(a: A, b: B):
        a.update(b)
    
    a: A = {}
    b: B = {'foo': 'a', 'bar': 'b'}
    f(a, b)
    print(a)

    It outputs {'foo': 'a', 'bar' : 'b'}. I recon this is a bug.

    lenards-speechmatics
    @lenards-speechmatics
    OK, I guess I found the update related github issues, nasty
    lenards-speechmatics
    @lenards-speechmatics
    should bool and Union[Literal[False], Literal[True]] be interchangeable in all contexts?
    3 replies
    George Farcasiu
    @georgeaf99

    Hi, I work on a large monolithic Python codebase at a corporation. I’m looking for advice on improving our mypy performance. Our codebase has many packages (with interdependencies)—some portions are typed. Like Dropbox, we have tangling issues, with large SCCs (strongly connected components) of >100 modules.

    I’ve tried out remote caching and dmypy (great for dev iteration loop), as suggested in “Our journey to type checking 4 million lines of Python” —both have helped, thank you! However, mypy still processes a large amount of “unused” code. In general, we would like mypy to gather as much type information as possible from external dependencies (e.g. another package), but only perform “expensive” analysis internally. Though I understand that isn’t always possible with Python.

    I’ve gathered some profiles of mypy runs—with py-spy and a custom profiling harness (with mypyc disabled). About 70% of runtime is spent in mypy.build.process_stale_scc, 20% in mypy.build.process_fresh_modules, and <10% in mypy.build.State.__init__. Most of process_stale_scc’s runtime is spent processing large SCCs. I’ve also noticed that even with a “remote cache” provided, process_stale_scc can be called on an unchanged SCC. Can an unchanged SCC with cached data be stale? Or perhaps there is a caching configuration issue?

    I appreciate all the performance work y’all have done so far! And don’t necessarily expect mypy to perform well in this use case—“untangling” and dependency management are the long term fixes. That said, do y’all have additional recommendations for improving performance? Could we configure mypy to only gather basic type information from external dependencies?

    Maarten Braakhekke
    @mbraakhekke

    I'm trying to subclass io.TextIOBase. Consider the following minimal example:

    from io import TextIOBase
    from typing import Optional
    
    class MyTextIOBase(TextIOBase):
        def readline(self, n:Optional[int]=-1) -> str:
            return "foo\n"
    
        def read(self, size:Optional[int]=1) -> str:
            return "foo"

    MyPy gives the following error for line 5: 'Return type "str" of "readline" incompatible with return type "bytes" in supertype "IOBase"'.
    The return type of readline() in IOBase is indeed bytes but I'm subclassing TextIOBase where the return type of readline() is str. Why is MyPy looking at IOBase? Moreover, for read() no error is given, while this has the same return type.

    lukas
    @mindrunner

    I have a problem where mypy does not detect a typing issue in our code. Trying to simplyfy the example here:

        def set_score(self, score: float) -> None:
            self._score_update_time = time.time()
            self._score = score
    
        def init_data(self) -> None:
            self.set_score(self.get_data('score'))
    
        def get_data(self, key: str) -> str | dict | None:
           return ....

    My IDE Pycharm warns me with Expected type 'float', got 'str | dict | None' instead, which is what I would assume.

    mypy stays silent and does not catch that potential bug.

    [mypy]
    warn_return_any = True
    warn_unused_configs = True
    strict = True
    ignore_missing_imports = True
    pretty = False
    check_untyped_defs =

    What am I missing here?

    2 replies
    lukas
    @mindrunner

    I am still struggling to make this work as expected. This is a semi large codebase with about 10 modules in the src-root. Each of them containing sub modules with sub-sub modules etc.

    My goal is to just type check everything because the plan is to add type annotations throughout the whole code. Now it looks like different ways of calling mypy giving me different outputs.

    For example:

    One module is called operations containing a monitor_operation.py (besides others).

    Calling mypy -p operations returns with Found 176 errors in 27 files (checked 33 source files)
    Calling mypy -p operations.monitor_operation returns with Found 68 errors in 6 files (checked 1 source file)

    which on the fist glance might make sense.

    However, the call operations lacks some errors which appear only when explicitly checking the monitor_operation with the latter call.

    Same for just running mypy src/ (Which is what I actually want)
    Found 171 errors in 38 files (checked 133 source files)

    What is going on here? Why is running mypy running on a bigger scope finding less errors than calling it on only a module?

    12 replies
    Alex Grönholm
    @agronholm

    Could somebody explain why this gives me an error with mypy?

    import asyncio
    from contextvars import copy_context
    
    
    async def run_task() -> None:
        pass
    
    
    def main() -> None:
        copy_context().run(asyncio.create_task, run_task())

    The error:

    blaa.py:10: error: Argument 2 to "run" of "Context" has incompatible type "Coroutine[Any, Any, None]"; expected "Union[Generator[Any, None, _T], Coroutine[Any, Any, _T]]"  [arg-type]

    If I change the return type to Any, it passes without error. Anything else? An error.

    1 reply
    Ronny
    @ronnypfannschmidt:matrix.org
    [m]
    anyone aware of any tool to automatically translate type annotations/a subset of one function/type/typed dict into the arguments/type annotations of another function/type
    minrk
    @minrk:matrix.org
    [m]

    Anyone have experience with typing use of optionally async APIs?

    I have a lot of success with the pattern:

    result = maybe_async_func() # e.g. int | Awaitable[int]
    if inspect.isawaitable(result):
        result = await result # Awaitable[int] -> int
    # at this point, `result` should be known to be `int`

    but I haven't figured out the best way to annotate for mypy to understand the type of result.

    2 replies
    lenards-speechmatics
    @lenards-speechmatics

    mypy --strict finds no issues in the following program:

    from typing import Optional
    
    class C:
        def __init__(self) -> None:
            self.x: Optional[int] = 0
    
    def foo(c: C) -> None:
        c.x = None
    
    def bar(c: C) -> int:
        if c.x is not None:
            foo(c)
            return c.x
        return 0
    
    c: C = C()
    x: int = bar(c)
    print(isinstance(x, int)) # prints False

    Is this something that can be considered to be a bug? Or is it an inherent limitation of type narrowing?

    9 replies
    karlicoss
    @karlicoss:matrix.org
    [m]

    Hi! I'm trying to achieve something... basically TLDR is there a way to hint mypy that YieldType is the same as SendType in Generator?

    from typing import Generator, Union
    
    Gen = Generator
    
    
    def fun() -> Gen:
        a = yield ' str '
        print(a.strip())
    
        b = yield 123
        b += 456
        print(b)
    
        c = yield "bad"
        c += 1111  # I want mypy to flag this as an error
    
    
    def run(g: Gen) -> None:
        # this function simply bounces value yielded by the generator back
        last_sent = None
        while True:
            try:
                x = g.send(last_sent) # type: ignore[arg-type]
            except StopIteration:
                break
            else:
                last_sent = x
    
    run(fun())

    https://mypy-play.net/?mypy=latest&python=3.10&gist=8c685b8459f3ce43c67bfdc0fe8eaca0

    ^^ this type checks, but misses the error on line 15 (not surprising since it's basically Any)

    I could type Gen like this:

    T = Union[int, str]
    Gen = Generator[T, T, None]

    but then I basically have to do casts/instance checks every time since mypy doesn't know which member of Union I receive back from run

    2 replies
    Karhide
    @Karhide
    Hey all, I'm sure this isn't possible out of the box but does anyone know if there's a nice way using ParamSpec to say function 'b' has the same arguments as function 'a' but the expected argument value types are individually modified somehow, e.g. they're all awaitable? Thanks!
    Karhide
    @Karhide
    Ah just found a relevant issue: https://github.com/python/typing/discussions/1163
    TLDR: Not possible yet :D
    Fabian Beitler
    @swamper123
    How would you type a parameter, which is either None or a concrete type of another abstract class?
    I want to implement a component based architecture and stuck here.
    Should be Python3.8 compatible
    
    class AbstractManagerClass(ABC):
    
        @abstractmethod
        def __init__(self, var1: str, config_manager: <What to write here for a concrete AbstractConfigManager type?>) -> None:
            """
            Constructor
            """
            super().__init__()
            self.v = var1
            self.config_manager = config_manager
    
    # .....
    
    class AbstractConfigManager(ABC):
    
        # some other abstract stuff
    2 replies
    Rafał Pitoń
    @rafalp
    I'm getting the "signature of serialize is incompatible with supertype" error for this code:
    GraphQLScalarSerializer = Callable[[Any], Any]
    
    
    class ScalarType:
        serialize: Optional[GraphQLScalarSerializer] = None
    
    
    class DateTimeScalar(ScalarType):
        @staticmethod
        def serialize(value: datetime) -> str:
            return value.isoformat()[:23] + "Z"
    and for the love of everything I have no idea how is it really incompatible?
    28 replies
    headtrick
    @headtrick:matrix.org
    [m]

    Dear friendly helpers.
    I'm struggling with some mypy/typing problems:

    X = TypeVar("X", bound=Union[str,int])
    
    def a(x: X) -> X:
        if isinstance(x, str):
            return x
        return x

    mypy complains: error: Incompatible return value type (got "str", expected "X")
    Is that a bug (For bound=str this works) or if not, how can I fix this?