Hi everyone,
I was wondering if people had experience using nlp.pipe to speed up pipeline processing. I have tried different batch size from 5,50,500,5000 to process 30000 texts with 1500 chars and have seen no speedups. I am on spacy 2.0.12 so is this just an issue with my library version or does batch processing not work?
I was wondering if people had experience using nlp.pipe to speed up pipeline processing. I have tried different batch size from 5,50,500,5000 to process 30000 texts with 1500 chars and have seen no speedups. I am on spacy 2.0.12 so is this just an issue with my library version or does batch processing not work?
Hi all, I have a question: How would you create answers for yes-no questions using spacy? Suppose we have a question like "Are you good today?", I would like to have something that automatically answers "Yes, I am good today.". I have tried multiple things but I didn't have any luck :/
I tried updating existing spacy ner model with my data, by now it is not able to detect even the GPE and other generic ones which it was able to do earlier, i know as mentioned it is forgetting it seems, what is the solution for it, I used 200 sentences with new entity types, those 200 sentences has only my new entity labelled data , should I missed on something, any suggessions
Is there an easy way to set up an EC2 server that works with spacy deep learning? I wanted to run through the spacy-transformers intro notebook, and thought using an Ubuntu deep learning AMI would work out of the box (+ installation of python packages using conda/pip of course) but ran into issue with using the GPU on a p2 machine ("GPU is not accessible. Was the library installed correctly?") . The cuda toolkit wasn't installed on that machine, but installing that via apt-get didn't do the trick either. Are there better AMI's to start with?
@sourabhmehtasg, Please have a look https://explosion.ai/blog/pseudo-rehearsal-catastrophic-forgetting
its about the size of training data that matters the most
to be more specific, calculating the saliency of a given entity from the entire document that was processed. I thought about using the similarity of the document with relation to the entity itself as a quick and dirty first attempt.
I am trying to set up new entity labels based on menu items, but I am running into issues with overlapping tokens. For instance, there are chicken wings and chicken strips - chicken is used optionally with wings. However, when I add
pattern1 = [{'LOWER': 'buffalo', 'OP': '?'}, {'LOWER': 'chicken', 'OP': '?'}, {'LOWER': 'wings'}]
pattern2 = [{'LOWER': 'chicken'}, {'LOWER': 'strips'}] - I get an error for overlapping tokens. How can I have the same token chicken be a part of multiple entities?
Hi, I have updated a spacy model with my new entity, now I am looking into its deployement part, any leads or help on how to deploy it, as I see when i save the new updated trained model, it is saved a folder structure inside main folder, now to use it I can load the main folder fully and use it, but now for productnising it, what should be the points I must consider, any guide or help will be nice, thanks
@shinde-rahul @shinde-rahul thanks
Hi, anyone here got the example from this blog post (https://explosion.ai/blog/spacy-transformers) working? It always throws the following error:
~/miniconda3/envs/dialog/lib/python3.7/site-packages/spacy_transformers/pipeline/tok2vec.py in set_annotations(self, docs, activations)
196 # to do in one shot without blowing up the memory?
197 for i, word_piece_slice in enumerate(wp_rows):
--> 198 doc.tensor[i] = wp_weighted[word_piece_slice].sum(0)
199 doc.user_hooks["vector"] = get_doc_vector_via_tensor
200 doc.user_span_hooks["vector"] = get_span_vector_via_tensor
cupy/core/core.pyx in cupy.core.core.ndarray.__getitem__()
cupy/core/_routines_indexing.pyx in cupy.core._routines_indexing._ndarray_getitem()
cupy/core/_routines_indexing.pyx in cupy.core._routines_indexing._prepare_slice_list()
IndexError: too many indices for array
Hello everyone. I'm trying to train a Spacy Entity Linking model using Wikidata and Wikipedia, using the scripts in https://github.com/explosion/spaCy/tree/master/bin/wiki_entity_linking. I've generated the KB and moved to training the model, but that is not done yet after more than a week. How long should that take normally? (I'm not using a GPU)
Is there a pretrained Wikidata entity linking model I can use?
Should I import codecs or something like that?
@davidbren Please see this video https://youtu.be/sqDHBH9IjRU, this has hints but me too have the same question
I have a token that is a left bracket, that was parsed from the sentence: [carlota] Chicas, ponedla aquí.
(Pdb) pp token
[If I check if it is a punctuation it says yes:
(Pdb) token.is_punct
TrueBut then I get the part of speech and it says PROPN not PUNCT:
(Pdb) token.pos_
'PROPN'
I would like to use FastText vectors in
nl_core_news_sm
so I can create a new model with
python3 -m spacy init-model nl ..., but then I won't have the other pipeline components like sentencizer, NER etc. in that new model
I'm in a position where my company is maintaining a fork of spaCy. I'm trying to determine how the build artifacts that are posted to PyPI are generated so that we can build them ourselves. The README explains how to do a local custom build. However, I am in need of posting to a private pip registry. The best I can determine is that it has something to do with the
fabfile.py file and the builds are generated and posted through the buildkite service. Can someone help me?
Upon closer look, it appears spaCy builds come from this project https://github.com/explosion/wheelwright
@gsgoncalves I never got the error you mentioned. I used the default parameters.
@Bipinoli I did not split on conjunctions inside spacy but did so in an iterator outside after creating a masterTokens list for each sentence. In my case, it was important to locate the predicate (single-predicate sentence) in order to spot triple structures around that predicate.
the model. For example bucketlist vs bucket list.
In theory the similarly for these two should be basically identical.
Is this something the tokenizer should be handling?
Hey spacy enthusiasts, is there an OR-operator for the matcher (except the IN-operator)? Or in other words: How can I include two words in an IN-operator? Example: I want to match also "two rabbits" in pattern = ({'LEMMA': {'IN': ["dog", "cat", "rat"]}} without creating a second pattern. Thanks
It would be good to have an estimate of the time (like time by number of items in the training set) required to train an entity linking. Anybody has got anything like that?
