I like Philip's idea of reflecting the confidence of the ML id prediction (e.g. by colour) in the identifier window. Is this data stored somewhere after running the algorithm?
the current algorithm does not directly give confidences in a meaningful way (which is a warning sign)
the -log likeihood of the matches themselves might give such a thing
Maybe this old MR plom/plom!1090 might give some hints? If you want to hack on the UI aspect, you could assume that "prediction_list.txt" has an extra "confidence" column say floats in [0, 1]
arechnitzer: it seems you are correct about threading: when someone refreshes manager (a slow operation on a large exam) my uploads pause for a few seconds. I suspect this is because of the
with atomic(): blocks, effectively holding a mutex---does that make sense?
Curious about how PLOM works under the hood. Apparently this requires reading lots of Python. So ... which is the top-level source file with the shortest path-length to a working msgr<->server link? Where is the API actually implemented? (If there is better documentation than "Read the source, Luke", please provide a pointer.) Thanks.
PS: No GUI please!
We're working toward Sphinx-generated docs like every other Python project has. Here's a sneak preview: https://personal.math.ubc.ca/~cbm/plom/doc/build/html/solutions.html#technical-documentation
(I linked to Solutions b/c that modules has a reasonably complete docs)
re: API: I think Joey Shi had a summary here: https://gitlab.com/plom/plom/-/issues/1879#note_847031556
Thanks. Good stuff here! Now in principle I am supposed to be working on other things today ... so I will do some reading and experimenting in the evening.
Hi everyone, apologies if this is documented somewhere (I've been trying to find it for a couple of days). I'm on macOS 12.3.x and I couldn't get the docker image to run properly (lack of experience), so I installed plom through pip3. I'm preparing a test using the 3 questions example shown on the docs. I got to https://plomgrading.org/docs/walkthrough/server.html and I'm stuck at the 'plom-server class' bit (invalid choice). It seems that I should go with 'plom-create make' (I don't have a classlist), but I'm unable to get it to work (I get this error and I'm unable to figure out how to proceed: "plom.plom_exceptions.PlomSSLError: HTTPSConnectionPool(host='0.0.0.0', port=41984): Max retries exceeded with url: /Version (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate (_ssl.c:1129)')))"). Recommendations?
I wonder if those docs are out of date: there is no "plom-server class" anymore I think... Once you get server running, you should use
plom-create class ... to upload a classlist. Its also possible to build tests without a classlist.
I've glanced over those docs: they are quite out of date: sorry for this! We're in a bit of a flux b/w versions right now
'plom-create uploadspec ...'
hi @beniltoncarvalho_gitlab did you get any further with it? Happy to help if I can
@cbm755:matrix.org , on my way to print out the exams! :-) couldn't be happier (but will get on the scanning adventure on the next few days)
You'll be the first real user outside of UBC... so I want to make sure its a good experience. We can take this to a private conversation cbm@math.ubc.ca
New version 0.9.0 is en-route: watch the "xmas tree" light up at https://gitlab.com/plom/plom/-/pipelines/548324284
No formal roadmap yet: that is a good idea! Auto-graded multiple choice is not supported, maybe someday, or via some scripting, especially if we had help to do it. Maybe we can follow-up via email over weekend or next week.
We'd be very excited to chat about this. Both Colin and I have done some thinking about automagical grading of MC... or at least "accelerated" grading of MC, but nothing has been put into code as yet... I used AMC quite a bit in the before-plom-times. In fact, I think I was introduced to AMC by ... actually - I think it might have been you!
Head's up CI is failing on all branches, macOS stuff waiting on gitlab-com/runner-saas-macos-access-requests#536 and some changes from new numpy here plom/plom#2250
1 reply
what's the recommended config file format. I am looking to describe to PLOM stuff about MC-question related information (which question/version/where the boxes are). Later this would ideally be harvested from a LaTeX file or through a GUI from a generic PDF. IN the meantime I'm going to handcode it and it might remain an internally kept config file. I see json and toml both used. Any preferences?