Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Nov 20 09:22
    josalhor closed #1123
  • Nov 20 09:22
    josalhor commented #1123
  • Nov 20 08:49
    wil93 commented #1123
  • Nov 19 16:53
    codecov[bot] commented #1143
  • Nov 19 16:52
    codecov[bot] commented #1143
  • Nov 19 16:43
    stefano-maggiolo edited #1143
  • Nov 19 16:43
    wil93 opened #1143
  • Nov 11 18:20
    edomora97 edited #1142
  • Nov 11 17:54
    codecov[bot] commented #1142
  • Nov 11 17:52
    codecov[bot] commented #1142
  • Nov 11 17:43
    stefano-maggiolo edited #1142
  • Nov 11 17:42
    edomora97 opened #1142
  • Oct 30 13:18
    codecov[bot] commented #1141
  • Oct 30 13:16
    codecov[bot] commented #1141
  • Oct 30 13:04
    codecov[bot] commented #1141
  • Oct 30 13:04
    edomora97 synchronize #1141
  • Oct 30 08:54
    codecov[bot] commented #1141
  • Oct 30 08:52
    codecov[bot] commented #1141
  • Oct 30 08:42
    codecov[bot] commented #1141
  • Oct 30 08:42
    edomora97 synchronize #1141
Andrey Vihrov
@andreyv
In the end I will likely just copy the plugin 🙂 This way it is much easier for someone to create a modified out-of-tree language plugin based on the official C++ language.
cfalas
@cfalas

Am I the only one having the issue of cms services stopping? All of them are closed, the SSH session I'm using to connect to it closes, but the system doesn't restart (or at least it shouldn't according to uptime (I'm using Ubuntu Server 18.04, cms v1.4.rc1)

For example, last night I was adding tasks until 9:00, and according to the log(https://paste.ubuntu.com/p/k2sbFtzbqn/), at 9:30 the services stop. Any ideas on what may be causing this? It has happened more than once (all of the times that this has happened have been after inactivity, but it may just be a coincidence)

cfalas
@cfalas
I think I found the cause behind this. It's because the SSH shell closes, and the cmsResourceService is still attached to it, the job also stops. Therefore, when I disown the service, it seems to work
Stefano Maggiolo
@stefano-maggiolo
The standard solution is to use screen or tmux
Andrey Vihrov
@andreyv
Ideally you should configure CMS as a system service, so that it is managed and restarted by the system
Andrey Vihrov
@andreyv
For reference, here are our service files for systemd: https://gist.github.com/andreyv/968d11b4ac4fb5c2543dc5ae776b8380
cfalas
@cfalas
Thanks. I ended up making a bash file that starts all cms services and used that for a systems service
Andrey Vihrov
@andreyv

Currently multiple programming language plugins such as Python and Java go to extra lengths to work with an extensionless executable file name. I would like to introduce a Language.executable_extension() method and use it to prepare executable names in task plugins. Somewhere here:
https://github.com/cms-dev/cms/blob/d4c9e926bd52d8022069c417b206b0882ef4d1ba/cms/grading/tasktypes/Batch.py#L172
In the end this would allow to simplify compilation/evaluation commands for multiple languages.

Does this sound good to work on?

Stefano Maggiolo
@stefano-maggiolo
yeah could be
you'll need to run updaters though
Andrey Vihrov
@andreyv
yeah, that's interesting
the updater shouldn't really call language plugins, because they might get refactored in the future
so it needs to have a hardcoded list of languages and executable extensions
Stefano Maggiolo
@stefano-maggiolo
yes
Phan Nhật Minh
@Nhatminh2442001_twitter

hello everyone, i have some problems when submit file, please help me

21:16:14 ERROR Worker,3
Worker failed: Failed to initialize sandbox with command: isolate --cg --box-id=42 --init (error 2).
21:16:14 ERROR EvaluationService,0
Received error from Worker (see above), job group lost.
21:16:14 ERROR EvaluationService,0
Worker,3 signaled RPC for method execute_job_group was unsuccessful: JobException: 'Worker failed: Failed to initialize sandbox with command: isolate --cg --box-id=42 --init (error 2).'
Traceback (most recent call last):
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/service/Worker.py", line 127, in execute_job_group
task_type.execute_job(job, self.file_cacher)
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/grading/tasktypes/abc.py", line 235, in execute_job
self.compile(job, file_cacher)
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/grading/tasktypes/Batch.py", line 231, in compile
sandbox = create_sandbox(file_cacher, name="compile")
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/grading/tasktypes/util.py", line 71, in create_sandbox
sandbox = Sandbox(file_cacher, name=name)
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/grading/Sandbox.py", line 944, in init
self.initialize_isolate()
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/grading/Sandbox.py", line 1420, in initialize_isolate
"(error %d)" % (pretty_print_cmdline(init_cmd), ret))
cms.grading.Sandbox.SandboxInterfaceException: Failed to initialize sandbox with command: isolate --cg --box-id=42 --init (error 2)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/io/rpc.py", line 411, in process_incoming_request
response["data"] = method(**request["data"])
File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cms/service/Worker.py", line 143, in execute_job_group
raise JobException(err_msg)
cms.grading.JobException: 'Worker failed: Failed to initialize sandbox with command: isolate --cg --box-id=42 --init (error 2).'

Stefano Maggiolo
@stefano-maggiolo
can you run isolate from the command line?
it might be that you didn't install it, or that your system has some problem running it
Phan Nhật Minh
@Nhatminh2442001_twitter

@stefano-maggiolo, i can run isolate with terminal
minhphan@DESKTOP-RPEUH8I:~$ isolate
Please specify an isolate command (e.g. --init, --run).
Usage: isolate [<options>] <command>

Options:
-b, --box-id=<id> When multiple sandboxes are used in parallel, each must get a unique ID
--cg Enable use of control groups
--cg-mem=<size> Limit memory usage of the control group to <size> KB
--cg-timing Time limits affects total run time of the control group
(this is turned on by default, use --no-cg-timing to turn off)
-c, --chdir=<dir> Change directory to <dir> before executing the program
-d, --dir=<dir> Make a directory <dir> visible inside the sandbox
--dir=<in>=<out> Make a directory <out> outside visible as <in> inside
--dir=<in>= Delete a previously defined directory rule (even a default one)
--dir=...:<opt> Specify options for a rule:
dev Allow access to special files
fs Mount a filesystem (e.g., --dir=/proc:proc:fs)
maybe Skip the rule if <out> does not exist
noexec Do not allow execution of binaries
rw Allow read-write access
-D, --no-default-dirs Do not add default directory rules
-f, --fsize=<size> Max size (in KB) of files that can be created
-E, --env=<var> Inherit the environment variable <var> from the parent process
-E, --env=<var>=<val> Set the environment variable <var> to <val>; unset it if <var> is empty
-x, --extra-time=<time> Set extra timeout, before which a timing-out program is not yet killed,
so that its real execution time is reported (seconds, fractions allowed)
-e, --full-env Inherit full environment of the parent process
--inherit-fds Inherit all file descriptors of the parent process
-m, --mem=<size> Limit address space to <size> KB
-M, --meta=<file> Output process information to <file> (name:value)
-q, --quota=<blk>,<ino> Set disk quota to <blk> blocks and <ino> inodes
--share-net Share network namespace with the parent process
-s, --silent Do not print status messages except for fatal errors
-k, --stack=<size> Limit stack size to <size> KB (default: 0=unlimited)
-r, --stderr=<file> Redirect stderr to <file>
--stderr-to-stdout Redirect stderr to stdout
-i, --stdin=<file> Redirect stdin from <file>
-o, --stdout=<file> Redirect stdout to <file>
-p, --processes[=<max>] Enable multiple processes (at most <max> of them); needs --cg
-t, --time=<time> Set run time limit (seconds, fractions allowed)
-v, --verbose Be verbose (use multiple times for even more verbosity)
-w, --wall-time=<time> Set wall clock time limit (seconds, fractions allowed)

Commands:
--init Initialize sandbox (and its control group when --cg is used)
--run -- <cmd> ... Run given command within sandbox
--cleanup Clean up sandbox
--version Display program version and configuration
i don't know what my problem is

Stefano Maggiolo
@stefano-maggiolo
Try the same command as the error (with --init)
Also, when you ran cms, it should have left a directory with the sandbox and some debugging output, you can try looking in that and see if there's anything useful
Should be under /tmp/ something
Phan Nhật Minh
@Nhatminh2442001_twitter
minhphan@DESKTOP-RPEUH8I:~$ isolate --init
/var/local/lib/isolate/0
is it normal ?
Phan Nhật Minh
@Nhatminh2442001_twitter
i found nothing in tmp/cms-compile-unf2s49w @@
Stefano Maggiolo
@stefano-maggiolo
yep that's normal
the other thing that this might be is that the user you use for CMS is not in the group that can run isolate as root
would be strange since mihphan can
but check, if you have a weird setup where you run CMS as another user
Andrey Vihrov
@andreyv
What is the use case for multiple submission files in Batch task type (and, I guess, in Communication as well)?
Phan Nhật Minh
@Nhatminh2442001_twitter
@stefano-maggiolo thanks, problem solved
but i found new problem with RWS
14:48:36 WARNING ProxyService,0 Status 400 while sending submissions to ranking http://usern4me:passw0rd@localhost:8890/.
Stefano Maggiolo
@stefano-maggiolo
@andreyv IIRC some tasks call for two functions in different files - not that it changes anything, they will just compiled together
@Nhatminh2442001_twitter care to share the solution for future readers?
for that error: I assume you have started RWS with the same arguments - if so, you should be able to see the error in the RWS logs
Phan Nhật Minh
@Nhatminh2442001_twitter
@stefano-maggiolo i used sol of 'https://github.com/cms-dev/con_test' to submit it, i dont't know what to do next
Stefano Maggiolo
@stefano-maggiolo
Phan Nhật Minh
@Nhatminh2442001_twitter
@stefano-maggiolo yes, i did but i think when i use default config and will not do anything
Stefano Maggiolo
@stefano-maggiolo
not sure I understand - if you launched RWS, when ProxyService gives you that error, do you see any logs in RWS?
if not, then there is probably a mismatch in the address
between the configuration of PS in cms.conf and that of RWS in cms.ranking.conf
Phan Nhật Minh
@Nhatminh2442001_twitter
2019-09-04 15:35:52.062  WRN  Invalid data.
    http://localhost:8890/submissions/

    {'5': {'task': 'batch_5ffile', 'time': 1567586085, 'user': 'u1'}}

    Exception cmsranking.Entity.InvalidData:

        [entity 5] Inconsistent data

    Traceback (most recent call last):
    File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cmsranking/RankingWebServer.py", line 216, in put_list
        self.store.merge_list(data)
      File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cmsranking/Store.py", line 256, in merge_list
        raise exc
      File "/home/minhphan/cms_venv/lib/python3.6/site-packages/cms-1.4rc1-py3.6.egg/cmsranking/Store.py", line 250, in merge_list
        raise InvalidData("Inconsistent data")

::ffff:127.0.0.1 - - [2019-09-04 15:35:52] "PUT /submissions/ HTTP/1.1" 400 303 0.001005
Stefano Maggiolo
@stefano-maggiolo
mmm
for some reason the current data PS wants to send is imcompatible with the existing data in RWS
the easiest way is to clear the data in RWS
Andrey Vihrov
@andreyv
@stefano-maggiolo Thanks. Regarding multiple sources, something doesn't seem right here: https://github.com/cms-dev/cms/blob/master/cms/grading/languages/python3_cpython.py#L62
Suppose the submission format is two files, hello.%l,extra.%l. If something in extra.%l tries to import hello, then it will break because the file was renamed to __main__.pyc.
Stefano Maggiolo
@stefano-maggiolo
oh well
YMMV :)
you're going into a case which is the intersection of many rare cases
  1. multi-file batch