Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Nik
    @nikudalov
    This message was deleted
    lamerman
    @lamerman
    @gnaresh I tried to find a simple way to get pid in subprocess but could not find it :( You can get pid of the /bin/sh but not the process that it runs. It is possible to traverse the tree of child processes of this sh, but it is not yet implemented.
    Things like limits for execution and memory at first glance should be set somehow inside of shell, not by shellpython, I think. So if there is a command that can limit memory it should be used then.
    lamerman
    @lamerman
    time mesurements should look like
    result = `time python compute.py
    so, not by means of shellpy but by means of other tools and internals of the script you run
    Naresh Ghanate
    @gnaresh
    Yes i got to know that. But it works efficiently only when there are small statements. If we have nested statements and you want to limit the time and memory of the script, it gives a problem. Lemme share a piece of code with you.
    Naresh Ghanate
    @gnaresh

    The file executor.py has the way to limit the memory usage of a subprocess. and using threads am able to limit the time for the subprocess. But the time is inefficient. It isn't stable for a subprocess for any 2 runs of the same command. To achieve this, I have to use /usr/bin/time command to get the exact time. and <bold>to limit time i have to have a shell script with threads in a line</bold>, rather than a subprocess call in python with threads. and the command I am using isn't simple as "python compute.py" it is something that is computed in this below specified form and is sent to subprocess.

    compile_cmd = pathofexecutable + " " + source_file + ' < ' + input_file + ' 1> ' + output_file + ' 2> ' + error_file
    main_command = /usr/bin/time -o timeoutputfile -f {'time':'%E', 'cpu':'%P', 'memory':'%M'} " + compile_cmd

    The first line creates a python/perl file runnable command
    the second line uses time command to get time and memory of the previous line

    The thing is I am unable to limit time here, I was able to limit memory using the "setrlimit" of resource for subprocess. But unable to limit time for the shell command accurately.

    FYI, I am on a verge to create a sphere-engine look a like.
    Naresh Ghanate
    @gnaresh

    @lamerman

    """
    also getting the pid of a subprocess is possible only when the subprocess is on a run. and if you have a small command which executed in micro second, you cannot get the subprocess pid. though if you use an infinite loop and then try to get pid it is possible. But the subprocess pid is the pid of the subprocess parent class object not the child shell process. Hence even if you get the pid it isn't efficient.

    """
    Nik
    @nikudalov

    Hi, @lamerman !

    Example of error traceback:

    Traceback (most recent call last):
      File "/tmp/shellpy_root/root/cr/scripts/monitor.py", line 272, in <module>
        some_error_line
    NameError: name 'some_error_line' is not defined

    But, actually, 'some_error_line' is line 262, not 272
    Is it possible "compile" to python with same lines numbers as in .spy file?

    lamerman
    @lamerman
    @nikudalov there is an issue for this, but it's not yet implemented and it is not clear whether it will be possible
    Nik
    @nikudalov
    Another question:
    Is it possible to correct shutdown program with "ctrl+c", for example?
    My usual traceback is:
    ^CTraceback (most recent call last):
      File "/tmp/shellpy_root/root/cr/scripts/monitor.py", line 272, in <module>
    Traceback (most recent call last):
      File "/usr/bin/shellpy", line 9, in <module>
        load_entry_point('shellpy==0.4.5', 'console_scripts', 'shellpy')()
      File "/usr/lib/python2.7/site-packages/shellpython/shellpy.py", line 40, in main
        main()
        retcode = subprocess.call(processed_file + ' ' + ' '.join(script_args), shell=True, env=new_env)
      File "/usr/lib64/python2.7/subprocess.py", line 524, in call
      File "/tmp/shellpy_root/root/cr/scripts/monitor.py", line 269, in main
        watchgod()
      File "/tmp/shellpy_root/root/cr/scripts/monitor.py", line 232, in watchgod
        return Popen(*popenargs, **kwargs).wait()
      File "/usr/lib64/python2.7/subprocess.py", line 1376, in wait
        sleep(30)
    KeyboardInterrupt
        pid, sts = _eintr_retry_call(os.waitpid, self.pid, 0)
      File "/usr/lib64/python2.7/subprocess.py", line 478, in _eintr_retry_call
        return func(*args)
    KeyboardInterrupt
    lamerman
    @lamerman
    How do you want it to look? :)
    Strange smiles here )
    Nik
    @nikudalov
    iphone style )
    lamerman
    @lamerman
    Yes possibly )
    @nikudalov the issue number is #2 for errors
    But as I said now it's not so clear how to make it
    Nik
    @nikudalov
    something like correct shutdown.
    I just execute sleep 10 command and interrupt it
    Oh, I see it now (#2)
    lamerman
    @lamerman
    Could you please paste the part of shellpy code where you interrupt it?
    As I understand it's monitor.py:272
    Ah no 232
    Nik
    @nikudalov
    while True:
        some_code_1()
        `sleep 30
        some_code_2()
    Something like this.
    lamerman
    @lamerman
    As I understand you want to remove from the stack trace everything that is above monitor.py
    Main of shell py and subprocess
    Is it correct?
    Nik
    @nikudalov
    Yes, because in general there is no errors )
    lamerman
    @lamerman
    Ok I think it is possible and reasonable, we could create an issue
    Either you or me a bit later
    Or it could be a part of task 2, just comment could be added there
    Nik
    @nikudalov
    But in any break you will get stack trace (not like bash-script style:)
    Looks like separate issue
    lamerman
    @lamerman
    Yes
    @nikudalov, btw as I understand you use shellpy. Could you share your thoughts? Is there something missing that definitely should be added? How does it work in general?
    Nik
    @nikudalov
    Yep, I already wrote 262 lines of shellpy code )
    Let me think about it and I'll write you a little bit later my opinion
    lamerman
    @lamerman
    ok, thank you
    Gurkirpal Singh
    @gpalsingh
    I found some typos on the wiki. In simple mode: Change "Compaire" to "Compare" and "latest" to something like "last one"
    Maxim Kolganov
    @MAnyKey
    @lamerman how did you manage to collect all that stars? :smile:
    lamerman
    @lamerman
    @MAnyKey with the help of God of course :)
    Юрий Рузыбаев
    @_Dark_Seraphim__twitter
    @lamerman is there any way to interactive use of this module like a vanilla python?
    lamerman
    @lamerman
    @_Dark_Seraphim__twitter , no, it's not implemented
    Юрий Рузыбаев
    @_Dark_Seraphim__twitter
    @lamerman will you implement this feature in near future?
    lamerman
    @lamerman
    @_Dark_Seraphim__twitter it was not planned for the near future :)
    Oliver Croomes
    @oliveratutexas
    Hi there
    Oliver Croomes
    @oliveratutexas
    Ahh, that was what I was going to ask.