Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Activity
    Snowapril
    @Snowapril
    Cool!
    Daniel Watkins
    @OddBloke
    RustPython/RustPython#3396 is relevant to exceptions, vm.ctx.new_classis used to create some of them.
    Daniel Watkins
    @OddBloke
    So, yeah, looking at RustPython/RustPython#3397 also, I suspect that they will now behave incorrectly with https://github.com/RustPython/RustPython/pull/3387/ because their __name__ is now set to module.name (rather than just name).
    And I didn't catch those when doing #3396, because we don't have a consistent way for creating exceptions.
    carbotaniuman
    @carbotaniuman
    @coolreader18 I was thinking a bit on your Object layout PR and about integrating GC/CC into everything
    carbotaniuman
    @carbotaniuman
    I think there's UB in this still, as WeakRefList needs to be collectively owned by all the weakrefs
    Noa
    @coolreader18
    WeakRefList has a weak reference to the weak references; when one weakref is dropped it removes itself from the list
    there actually isn't a super clear ownership pattern between them, like I was gonna say that weakreflist owns the weakrefs but they sort of just all exist in a web together?
    Like it is a linked list I suppose
    The one thing I am confused about wrt UB is trying to run rustpython on miri gives an error the first time an object is ever fully deallocated, which confused me cause afaict the refcounting is sound
    carbotaniuman
    @carbotaniuman
    What's the miri error you're getting?
    Noa
    @coolreader18
    error: Undefined Behavior: deallocating while item is protected: [Unique for <4552491> (call 1387536)]
       --> /home/coolreader18/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:104:14
        |
    104 |     unsafe { __rust_dealloc(ptr, layout.size(), layout.align()) }
        |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ deallocating while item is protected: [Unique for <4552491> (call 1387536)]
        |
        = help: this indicates a potential bug in the program: it performed an invalid operation, but the rules it violated are still experimental
        = help: see https://github.com/rust-lang/unsafe-code-guidelines/blob/master/wip/stacked-borrows.md for further information
    
        = note: inside `std::alloc::dealloc` at /home/coolreader18/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:104:14
        = note: inside `<std::alloc::Global as std::alloc::Allocator>::deallocate` at /home/coolreader18/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:239:22
        = note: inside `alloc::alloc::box_free::<rustpython::rustpython_vm::pyobjectrc::PyInner<rustpython::rustpython_vm::builtins::PyStr>, std::alloc::Global>` at /home/coolreader18/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/alloc/src/alloc.rs:333:9
        = note: inside `std::ptr::drop_in_place::<std::boxed::Box<rustpython::rustpython_vm::pyobjectrc::PyInner<rustpython::rustpython_vm::builtins::PyStr>>> - shim(Some(std::boxed::Box<rustpython::rustpython_vm::pyobjectrc::PyInner<rustpython::rustpython_vm::builtins::PyStr>>))` at /home/coolreader18/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
        = note: inside `rustpython::rustpython_vm::pyobjectrc::drop_dealloc_obj::<rustpython::rustpython_vm::builtins::PyStr>` at /home/coolreader18/projects/rustpython/vm/src/pyobjectrc.rs:64:57
        = note: inside `rustpython::rustpython_vm::PyObjectRef::drop_slow` at /home/coolreader18/projects/rustpython/vm/src/pyobjectrc.rs:683:18
        = note: inside `<rustpython::rustpython_vm::PyObjectRef as std::ops::Drop>::drop` at /home/coolreader18/projects/rustpython/vm/src/pyobjectrc.rs:746:13
        = note: inside `std::ptr::drop_in_place::<rustpython::rustpython_vm::PyObjectRef> - shim(Some(rustpython::rustpython_vm::PyObjectRef))` at /home/coolreader18/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
        = note: inside `std::ptr::drop_in_place::<std::option::Option<rustpython::rustpython_vm::PyObjectRef>> - shim(Some(std::option::Option<rustpython::rustpython_vm::PyObjectRef>))` at /home/coolreader18/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/ptr/mod.rs:188:1
        = note: inside `rustpython::rustpython_vm::builtins::PyType::set_str_attr::<rustpython::rustpython_vm::PyRef<rustpython::rustpython_vm::builtins::PyGetSet>>` at /home/coolreader18/projects/rustpython/vm/src/builtins/pytype.rs:157:56
        = note: inside `rustpython::rustpython_vm::builtins::property::init` at /home/coolreader18/projects/rustpython/vm/src/macros.rs:41:13
        = note: inside `rustpython::rustpython_vm::types::zoo::TypeZoo::extend` at /home/coolreader18/projects/rustpython/vm/src/types/zoo.rs:193:9
        = note: inside `rustpython::rustpython_vm::PyContext::init` at /home/coolreader18/projects/rustpython/vm/src/pyobject.rs:144:9
    could just be being pedantic - one of those "oh this is actually invalid under stacked borrows" things where you just need to shuffle some accesses around, but idk where that would be needed in the code since the deallocation is fairly straightforward
    carbotaniuman
    @carbotaniuman

    I'm pretty sure WeakRefList needs to be a side-table ala ObjC, or else it can be deallocated while the destructor is still running, but after you've grabbed the pointer.

    https://github.com/RustPython/RustPython/blob/6227ed455303fa43f7cca6b9cbde5a8144ee51b7/vm/src/pyobjectrc.rs#L280

    Noa
    @coolreader18
    I should probably add comments, and also probably run it under miri or set up loom/threadsanitizer, but I did think about that and I think the locking should ensure that if that does happen, it's ok - we check parent again after we get the lock, and nope I see what you mean. In theory it could run the entirety of WeakRefList::drop between load() and lock() ? Even unlock_fair wouldn't necessarily totally fix it, cause there could be a bunch of weakrefs queueing to drop and idk if unlock_fair would go to all of them before returning to the lock in WeakRefList::drop
    Noa
    @coolreader18
    I mean, I suppose parent could be a PyMutex
    Ahh, I see what you mean by side-table, that definitely looks like a much less error-prone solution
    Hoss Ajallooiean
    @katetsu
    im learning about rustpython and have looked at the embedded examples. are those examples embedding the kernel statically?
    also, if i go ahead with using rustpython, is there a limitation involved with pip installing other packages on pypi?
    does the c-api play a role here?
    Jeong YunWon
    @youknowone
    pip works in linux but any c-api related stuff doesn’t work (yet)
    Hoss Ajallooiean
    @katetsu
    Ok thanks
    Snowapril
    @Snowapril

    @coolreader18 I'm trying to implements "Parenthesized context managers(bpo-12782)" before adding RustPython/RustPython#3427 but stuck in lalrpop..

    What I'm trying to add is just support parenthesizing with context like below which is illegal in before.

    with (
                manager() as x,
                manager() as y
            ):
                pass

    But I'm struggling with shift-reduce conflict 😢
    Could you take a look and advice these lines

    Corresponding parts Python.gram in cpython are these lines

    Jim Fasarakis-Hilliard
    @DimitrisJim
    I remember seeing this as a rationale for introducing the PEG parser in CPython. Don't remember the PEP atm but should be easy to find.
    Jim Fasarakis-Hilliard
    @DimitrisJim
    This doesn't block RustPython/RustPython#3427 right?
    Snowapril
    @Snowapril
    Yeah. It doesnt matter with #3427. There are a lot bunch of lalrpop stuffs for pattern-matching. I think it will be long-term works. Before going them I want to quick-solve that bpo-12782 for practice. Thanks 😊
    Yossi Konstantinovsky
    @yossi-k
    Why doesn't OptionalArg allow NoneType? If I have a function that accepts OptionalArg<PyStrRef>, I want to be able to pass None as well as strings or not pass it at all
    Noa
    @coolreader18
    @yossi-k That's OptionalOption, aka OptionalArg<Option<T>>
    Snowapril
    @Snowapril
    It seems #3402, #3421, #3433, #3434 suffered from same test failure in test_cpython_stdlib_os 😢
    hohh
    @hohh:matrix.org
    [m]
    What is the status of the python-vm for rust? I'd like to have some embedded python in a rust application but I see it's been retracted from crates.io and there is a short note on it on the github.
    Steve Shi
    @qingshi163
    stdlib_os failing is not related with these pr, it looks like a version problem happened only on Windows. I don't have the Windows environment to test the code, but it not looks something important right now.
    Snowapril
    @Snowapril
    Yeah I also agree 😊. It was not relevant to those PR but because of it, those PR's test checks were stucked
    hohh
    @hohh:matrix.org
    [m]
    Does anyone know why rustpython-vm was yanked from crates.io?
    There is a note in the README that it's due to it not compiling with newer versions of rustc, however that seems like a poor reason to yank an entire crate. What is required to get this back?
    Jeong YunWon
    @youknowone
    I wrote a new guideline to update test files: https://github.com/RustPython/RustPython/wiki/How-to-update-test-files
    Zomatree
    @Zomatree
    is there any way to debug a memory leak inside rustpython?
    Gerald Nash
    @aunyks
    Hi!
    Does RustPython support module import / export for non-stdlib modules?
    I couldn’t tell from the docs or examples so thought I’d ask
    Jeong YunWon
    @youknowone
    what you mean by module export? if you are looking for python’s import equivalent thing, search for vm.import in the source code
    @Zomatree inside meaning python way? then I have no idea. in rust side, idk well but MIRI looks like the one
    Gerald Nash
    @aunyks
    @youknowone Yea thats what i meant. I wasnt sure how imports are resolved. Thanks ill check
    Snowapril
    @Snowapril
    How do you think to introduce continuous fuzz testing to RustPython? I found some good fuzz testing tool for open source software (https://github.com/google/oss-fuzz). It seems that many of major open source project(cpython, django, etc.. ) registered on it.
    Jeong YunWon
    @youknowone
    If you are interested in, rust is well-known for its advanced fuzz test tools https://rust-fuzz.github.io/book/introduction.html
    Gentle
    @Gentle
    can rustpython work with virtualenvs?
    hohh
    @hohh:matrix.org
    [m]

    I'm using the rustpython-vm crate in order to run some python code from a rust application.
    Currently I'm having issues with first compiling a python library, loading it into a module and after that calling functions from that module in a later call to the VM.
    I'm using the import_file function to create the module, but I'm not sure how to later create a scope for the VM that includes this module.

    Does anyone know how I could go about debugging this?

    use rustpython_vm::compile::Mode;
    use rustpython_vm::Interpreter;
    use rustpython_vm::import::import_file;
    use rustpython_vm::compile;
    
    fn main() {
        Interpreter::default().enter(|vm| {
            let imp = import_file(vm, "mylib", "mylib".to_owned(), MAIN.to_owned()).unwrap();
            //let import = vm.compile(MAIN, compile::Mode::Exec, "<main>".to_owned())
            //    .map_err(|err| vm.new_syntax_error(&err)).unwrap();
    
            let scope = vm.new_scope_with_builtins();
            let code_obj = vm
                .compile(
                    r#"import * from mylib
    def initt():
        print("fef")
    init()"#,
                    Mode::Exec,
                    "<embedded>".to_owned(),
                )
                .map_err(|err| vm.new_syntax_error(&err))
                .unwrap();
            //vm.run_code_obj(import, scope).unwrap();
            vm.run_code_obj(code_obj, scope).unwrap();
        });
        println!("Hello, rust!");
    }
    
    const MAIN: &str = r#"print("PYTHON MODULE INIT")
    def init():
        print("Python says init")
    
    def run(command, ext_state, state):
        print("Python says run")
    "#;
    This is a small reproduction of what I'm trying to do, notice how I don't know how to make the code_obj run in a scope that includes imp