Please make issues if you find anything you think is broken!
I'm Siddhant Sharma, I'm a third year student at IIIT-bhubaneswar and I'm a data science enthusiast. I want to contribute to the project, where should I start?
Hi @iamapickle ,
To understand the things, docs should be your friend :)
Documentation for the latest release - https://intel.github.io/dffml/
Documentation for the master branch - https://intel.github.io/dffml/master/index.html
Start looking in issues - https://github.com/intel/dffml/issues and try to solve "good first issue".
@pdxjohnny Hi all, it is jacob who intends to contribute to this project as a senior at NYU majoring in Computer Science. I was reading all the docs (I am still doing that!!), and I am watching for the code that leads me to be curious if there is still a plan to supply more "scratch" machine learning algorithms. I am seeing only slr model, but there could be more models to be supplied. May I supply more models for the dffml scratch models? I only worry that it is not your plan and my commit might make any confusion or problem. Thanks!!!
@jacob771 Of course! We're always looking for new models. However, we are particularly interested in implementations that do not hold the entire dataset in memory as they train the model. Many of the existing models (with the exception of daal4py) pull the entire dataset into memory, and we are trying to get away from that.
@pdxjohnny , Yay its been a long time since we had a release :smiley: I watched the video clip, so what we need is we need to fixup all the commits of a
particular subsystem into a single commit such as one given below all the commits of the transformer model should be a single commit, and similarly for other subsystems right ??
particular subsystem into a single commit such as one given below all the commits of the transformer model should be a single commit, and similarly for other subsystems right ??
7d76b560 model: transformers: accuracy: Use mctx instead of self
4af8fc2b model: transformers: accuracy: Add the init file
e2097a27 model: transformers: accuracy: Add the qa scorer
1e431e1f model: transformers: accuracy: Add the ner scorer
ffc67888 model: transformers: accuracy: Add the classification scorer
d300d261 model: transformers: qa: Add the correct scorer
315cd5c1 model: transformers: qa: Add the tqa scorer
45228c14 model: transformers: ner: Add the correct scorer
49761ac5 model: transformers: ner: Add the tner scorer
b6450a73 model: transformers: classification: Add the correct scorer
95818d18 model: transformers: classification: Add the tclf scorer
0365a4c2 model: transformers: setup: Add the scorers
6690b3cd model: transformers: classification: Change the scorer
5f164760 model: transformers: classification: add the method to modelcontext
This message was deleted
Let's see it without fixups, just rearranging first.
And then maybe fixup the commits made to the same file.
Greetings everyone. I am Aryan Kenchappagol currently pursuing my 2nd year computer science engineering from International Institute of Information Technology Pune. I was going through python/ML related repositories and I found DFFML pretty interesting. I am a beginner to open source and would love to work on this repository.
Hey @aryanxk02,
I would suggest you to start by studying the documentation.
Read the contributing guide to learn how to set up and start contributing to DFFML.
Once you have a good idea how things work, you can start working on good first issues which are curated to help beginners get a kick-start :)
Later, you can make your way through the rest of the documentation to understand the code base better.
Good Luck!
I would suggest you to start by studying the documentation.
Read the contributing guide to learn how to set up and start contributing to DFFML.
Once you have a good idea how things work, you can start working on good first issues which are curated to help beginners get a kick-start :)
Later, you can make your way through the rest of the documentation to understand the code base better.
Good Luck!
issue : intel/dffml#1007
I think I can solve this. Shall I make a PR??
I think I can solve this. Shall I make a PR??
@pdxjohnny As a follow-up question from your previous answer, I am first intending to modify logistic regression model on scratch folder that already existed. I am intending to change algorithm to be more efficient, and one effort that I made (I made a PR on it) is to use "from scipy.stats import multivariate_normal as mvn" to generate dataset rather than taking dataset.sh that was prepared. My intention is to build logistic regression model from scratch into more self-sustained, but I hope your criticisms to understand if there are problems that I missed. And you previously mentioned that you wish not to take entire dataset into the memory while training the dataset, and it is my idea that we could generate "random dataset" rather than taking from dataset.sh.
Hello everyone!
This is Sanjiban Sengupta, a junior undergrad from IIIT Bhubaneswar, India, pursuing Computer Engineering. I would like to contribute to dffml.
I started with solving the issue: intel/dffml#1002
and made the PR intel/dffml#1005
For the issue, I have commented out the property decorator for the key in Record class and added the self.key= str(key) in the init function.
Currently, 8 checks are failing, some of which states having this issue within the tests,
This is Sanjiban Sengupta, a junior undergrad from IIIT Bhubaneswar, India, pursuing Computer Engineering. I would like to contribute to dffml.
I started with solving the issue: intel/dffml#1002
and made the PR intel/dffml#1005
For the issue, I have commented out the property decorator for the key in Record class and added the self.key= str(key) in the init function.
Currently, 8 checks are failing, some of which states having this issue within the tests,
File "/home/runner/work/dffml/dffml/model/daal4py/tests/test_lr.py", line 77, in test_02_predict
correct = self.test_data[i]["Y"]
TypeError: list indices must be integers or slices, not str
Hey @kahanikaar, since you are forcing the key to be str and the tests expect the key(the
So the
i in this case) to be int, it throws the error. Perhaps a bit redundant, but you could handle this by just casting the type back into int, in the tests that are failing.So the
/home/runner/work/dffml/dffml/model/daal4py/tests/test_lr.py", line 77 should be changed to: correct = self.test_data[int(i)]["Y"] and similarly in the model/scratch failing tests.
5 replies
Help Needed! while installing dffml-model-pytorch; I'm getting an error ERROR: Could not find a version that satisfies the requirement torch==1.7.1+cu110; platform_system == "Windows" and python_version == "3.7" (from dffml-model-pytorch)
ERROR: No matching distribution found for torch==1.7.1+cu110; platform_system == "Windows" and python_version == "3.7"
ERROR: No matching distribution found for torch==1.7.1+cu110; platform_system == "Windows" and python_version == "3.7"
how to resolve it?
We should document this. Can you open an issue please?
2 replies
What is cat and EOF ??
Eof is end of file delimiter
Hi all can anyone help me why I am getting the following error in the ci test checks https://github.com/intel/dffml/pull/1015/checks?check_run_id=1967979265
I the following pull I created a new test case , for that I created a file in the
test directory called test_styles
And then I inherited the base class
IntegrationCLITestCase
Hey @Pratikrocks , the failing tests on your PR can be fixed by:
1: Adding an entry in the changelog.
2: Using black to style your code as it is the format of the whole code base.
You can find more about these CI tests and fixes in the 'Contributing -> Git and Github -> How to Read the CI' documentation section.
1: Adding an entry in the changelog.
2: Using black to style your code as it is the format of the whole code base.
You can find more about these CI tests and fixes in the 'Contributing -> Git and Github -> How to Read the CI' documentation section.
Hello, I have a flask web app with an upload page where I can upload a train and a test csv file that I use to train a model and then redirect to a "trained" page that shows the accuracy. The issue is that when i go back to the upload page and upload a different set of csv files for a different problem, while the model trains, the accuracy method returns a
NoRecordsWithMatchingFeatures exception, as if the test.csv file was never changed. I checked if the files are being uploaded correctly, which they are, overwriting the old files. The train function seems to work fine but I can't seem to get the accuracy function to work unless I either restart the server, or name the test file differently (like test2.csv for example) which I don't want to. Is there any way to get this to work fine? I believe this wasn't happening while using dffml 0.3.7
I was looking at the
source of the csv file I came cross the following config params https://github.com/intel/dffml/blob/38ddb55ce0276b4590a9afc89dad47ba8cab1e5e/dffml/source/csv.py#L42 Can anyone who have worked on it give me a jist of what role does the following CSV_SOURCE_CONFIG_DEFAULT_tag CSV_SOURCE_CONFIG_DEFAULT_KEY params does?