discussion on how we can use statistical methods to measure and improve the efficacy of http://freeCodeCamp.com
Hi, I'm pretty new to ML and trying to find out if my use case is already implemented and available to use.
Specifically I'd search something js based to use with node, like brain.js or tensorflow-js.
Im looking for a specific usecase to implement: I have a text and want to extract multiple specifications from that.
Probably it will use LSTM, but I would need multiple outputs for my input
It would be awesome if it could work in such a way:
Input: "Intel Core i7-7500U 2,70 GHz, 16 GB DDR4"
Output: { cpuManufactor: "Intel", cpuClockRate: "2,70", "memorySize": 16, "memoryType": "DDR4" }
The input wouln't always have the same length or contain the same information.
If it wouldn't work this way I could settle for a less dynamic pattern like:
Output: { cpuManufactorIntel: 1, cpuClockRateIs270: 1, memorySizeIs16: 1 memoryTypeIsDDR4: 1 }
However, getting near the first example would be awesome.
As the concept behind this text classification is pretty general and not related to my specific data I believe something similar to this must be implemented somewhere already.
However I didn't really find anything yet.
Thanks in advance for your help!
IMHO this should work without a dictionary and without actual language processing.
Just that I have enough training data similar to "Intel Core i7-7500U 2,70 GHz, 16 GB DDR4" resulting in { cpuClockRate: "2,70", "memorySize": 16 }
or similar examples like "Intel Core i7-2500U 1,80 GHz, 8 GB DDR4" resulting in { cpuClockRate: "1,80", "memorySize": 8 } so new Entries could be matched with a certain similarity
Just a reminder:
"Premature optimization is the root of all evil."
-- Donald Knuth
But thanks for the hint, I'll ingestivate how far regex matching can get me here.
Still, I feel like I might have some misconcept regarding the way some ML patterns work, as I'm quite baffled that my use case is neither a typical one nor (relatively) easy to achieve.
@janus-reith my rule of thumb on when to use deep learning is for tasks that are easy for humans to do but difficult to tell someone to do.
For example, if you think of all the typical deep learning applications, all of them are difficult to just tell someone about, like driving a car or producing art/images. But I could tell someone how to look for computer memory with some simple rules like, "If you see a number in front of 'GB' that is next to some letters like 'DDR', then use that number for memory size."
And again, the way technical specifications for computers is fairly standardized. The only thing you might have to worry about it using a comma instead of a decimal point in those numbers (1,6 versus 1.6).
However, explaining the logic to match all these patterns would be more difficult to explain to a human, while it would be easy for a human to understand the pattern, even without knowing the language.
Still I don't really get how this is not a designated task for ML
@janus-reith machine learning is generally divided into two categories: supervised and unsupervised. Supervised is you have labels on data and you want to correctly assign that label to that data. In unsupervised, you're just searching for patterns. Your problem is maybe closest to supervised learning.
You've mentioned that input would be something like
"Intel Core i7-7500U 2,70 GHz, 16 GB DDR4"and the example output would be
{ cpuClockRate: "2,70", "memorySize": 16 }This is more clearly just normal text processing because you are simply extracting information from a set of text. In other words, the answer you're looking for is within your input data.
An appropriately used machine learning task is spam filtering in emails. There are words within the emails that hint to you that it is spam, but the task is to categorize the data rather than simply extract weird words from an email. ML is also necessary because the number and types of words you may see in emails is unconstrained (i.e., you don't know all the words someone might use in an email).
Here are some tips from Amazon on when to use ML:
- You cannot code the rules e.g., spam or not spam. If you start seeing yourself writing a lot of rules to solve your problem, ML might help.
- You cannot scale to manually review by a human.
Again, information extraction for your products you should be able to code most if not all the rules. And the number of samples will depend on your situation.
import * as use from "@tensorflow-models/universal-sentence-encoder";
^
SyntaxError: Unexpected token *
at Module._compile (internal/modules/cjs/loader.js:721:23)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:787:10)
at Module.load (internal/modules/cjs/loader.js:653:32)
at tryModuleLoad (internal/modules/cjs/loader.js:593:12)
at Function.Module._load (internal/modules/cjs/loader.js:585:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:829:12)
at startup (internal/bootstrap/node.js:283:19)
at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)
