These are chat archives for lamerman/shellpy

1st
Mar 2016
Naresh Ghanate
@gnaresh
Mar 01 2016 05:52
Yes i got to know that. But it works efficiently only when there are small statements. If we have nested statements and you want to limit the time and memory of the script, it gives a problem. Lemme share a piece of code with you.
Naresh Ghanate
@gnaresh
Mar 01 2016 06:03

The file executor.py has the way to limit the memory usage of a subprocess. and using threads am able to limit the time for the subprocess. But the time is inefficient. It isn't stable for a subprocess for any 2 runs of the same command. To achieve this, I have to use /usr/bin/time command to get the exact time. and <bold>to limit time i have to have a shell script with threads in a line</bold>, rather than a subprocess call in python with threads. and the command I am using isn't simple as "python compute.py" it is something that is computed in this below specified form and is sent to subprocess.

compile_cmd = pathofexecutable + " " + source_file + ' < ' + input_file + ' 1> ' + output_file + ' 2> ' + error_file
main_command = /usr/bin/time -o timeoutputfile -f {'time':'%E', 'cpu':'%P', 'memory':'%M'} " + compile_cmd

The first line creates a python/perl file runnable command
the second line uses time command to get time and memory of the previous line

The thing is I am unable to limit time here, I was able to limit memory using the "setrlimit" of resource for subprocess. But unable to limit time for the shell command accurately.

FYI, I am on a verge to create a sphere-engine look a like.
Naresh Ghanate
@gnaresh
Mar 01 2016 06:09

@lamerman

"""
also getting the pid of a subprocess is possible only when the subprocess is on a run. and if you have a small command which executed in micro second, you cannot get the subprocess pid. though if you use an infinite loop and then try to get pid it is possible. But the subprocess pid is the pid of the subprocess parent class object not the child shell process. Hence even if you get the pid it isn't efficient.

"""