commonsense/conceptnet5

amirouche

@amirouche

I got something, with a mix a simhash and prefix lookup:

~/src/python/hoply/examples/conceptnet.io$ python query.py /media/amirouche/data/wt-hoply-fuzzyhash/ "concpt car" 10
concept car
concept cars
concept art
concept map
conceptual
conciliar
concept maps
conceptacle
concatemer
concatamer

and the query time is around 1s

Here is another example:

~/src/python/hoply/examples/conceptnet.io$ time python query.py /media/amirouche/data/wt-hoply-fuzzyhash/ "lulabby" 10
lullaby
luby
lunary
lualaba
luafy
lula
lunacy
lully
lulzy
luray

real    0m2.714s
user    0m1.655s
sys    0m1.018s

amirouche

@amirouche

Actually, the query takes around 100ms the rest of the time in spent in cpython startup and database connection handling

not bad.

amirouche

@amirouche

The code is at:

amirouche

@amirouche

actually fuzzyhash is funny but not very useful. Doing a prefix search then computing levenshtein distance is much faster

yyHaker

@yyHaker

Hi, is there a relation template which can be used to transfer a triple into a natural language in ConceptNet?

Chris Hart

@chrstphrhrt

I'm testing out ConceptNet for doing query expansion with synonyms etc., and ideally also for POS tagging since it seems to know this stuff too.. however, since I have short phrases (maybe 30 words at a time), would it be possible to do "bulk" queries somehow (either via the api or directly w/ python)?

amirouche

@amirouche

@chrstphrhrt you want to link words from sentences against concept from conceptnet?

Chris Hart

@chrstphrhrt

@amirouche yeah

amirouche

@amirouche

@chrstphrhrt try this https://stackoverflow.com/questions/52046394/how-to-perform-entity-linking-to-local-knowledge-graph

Kwame-NrD

@Kwame-NrD

Hello, I'm interested in the features that ConceptNet provides for a given concept and I would like to know if there is any way to harvest only features which are perceptual ?

Robyn Speer

@rspeer

@Kwame-NrD Can you clarify what would count as perceptual?

Kwame-NrD

@Kwame-NrD

I'm interested in the visual properties such as size, shape, parts .. for example : "airplane : is large,has windows, has wheels, has wings, has seats .. etc" .. I have used the API to attempt to extract similar features from ConceptNet by specifying the "HasA;HasProperty & MadeOf " relations, however, It doesn't always yield what I hope for. Are you familiar with any publicly available visual dictionaries that could do the task ?
Thanks in advance for taking the time to answer my questions.

Shen Guangyao

@sunlightsgy

The build process has not been over after one day. It stuck in here, is it normal?

Finished job 5.
470 of 472 steps (100%) done

Robyn Speer

@rspeer

I'd have to know which steps are unfinished, not the number of the job that was finished.

This is of course not normal, and I wonder if it's because of computational resource demands.

Shen Guangyao

@sunlightsgy

Already solved

Robyn Speer

@rspeer

What was the solution, or what changed?

Shen Guangyao

@sunlightsgy

Indeed I don't know. Just kill the process, and restart, and everything goes well...

Kaiqiang Huang

@kaiqiangh

Hi all, I downloaded ConceptNet5.7 version. How do I select English-only nodes and edges in a graph? Thanks.

amirouche

@amirouche

@kaiqiangh you can filter using concept.startswith("/c/en/")

edge are common to all languages

Kaiqiang Huang

@kaiqiangh

@amirouche Thanks for your help. That's a good way to do that. By the way, is it possible to filter English nodes in Concept 5.7 version (10GB) to generate new file?

Roman Inflianskas

@rominf

@kaiqiangh Are you talking about filtering the dump file to get the dump of smaller size?

Kaiqiang Huang

@kaiqiangh

Hi Roman, not sure what dump file is. What I did is that I downloaded conceptNet version from https://github.com/commonsense/conceptnet5/wiki/Downloads, and I try to only keep English-nodes in a graph, remove other language-node in order to get a smaller size file.

This file is a pre-built list of all the edges (assertions) in ConceptNet

Roman Inflianskas

@rominf

Ok, this is what I called dump. I think that the easiest way is to write a short Python script which reads the file line by line, parses the string and writes into another file only the required lines.

Kaiqiang Huang

@kaiqiangh

Thanks for your tips! I am gonna try it.

Roman Inflianskas

@rominf

@kaiqiangh You can make use this technique: https://stackoverflow.com/a/5463419/2108548

Kaiqiang Huang

@kaiqiangh

That's really what I need. Thank you so much!

Kaiqiang Huang

@kaiqiangh

Hi @rominf , sorry to ask you one more question. I got a subgraph with English-only nodes (around 3.4 million edges). Does this graph have some potential symmetric edges? As I checked the paper (ConceptNet 5.5: An Open Multilingual Graph of General Knowledge), it says the ConcepNet's edges are directed and some relations are designed as being symmetric (Antonym, DistinctFrom, EtymologicallyRelatedTo, LocatedNear, RelatedTo, SimilarTo, and Synonym).

For example, if I find an edge that A -> B with “RelatedTo” relationships, is it possible to find another edge that B -> A with “RelatedTo” relationships as well? Thanks in advance.

Roman Inflianskas

@rominf

@kaiqiangh

Hi @rominf , sorry to ask you one more question.

No problem.

I got a subgraph with English-only nodes (around 3.4 million edges).

Nice!

Does this graph have some potential symmetric edges? As I checked the paper (ConceptNet 5.5: An Open Multilingual Graph of General Knowledge), it says the ConcepNet's edges are directed and some relations are designed as being symmetric (Antonym, DistinctFrom, EtymologicallyRelatedTo, LocatedNear, RelatedTo, SimilarTo, and Synonym).
For example, if I find an edge that A -> B with “RelatedTo” relationships, is it possible to find another edge that B -> A with “RelatedTo” relationships as well? Thanks in advance.

Yes, at least because of the bugs. Here is what I found recently (in both: online service and the dump).

@rspeer Should I fill a bug report about this? ^^

Kaiqiang Huang

@kaiqiangh

@rominf Thanks for your reply again. I checked it just now. There have 40K - 50K edges like A->A (Self-loop), but I did not find the edge like A->B and B -> A with the same relation (not sure I'm correct or not)

Roman Inflianskas

@rominf

@kaiqiangh Thank you for sharing!

Kaiqiang Huang

@kaiqiangh

@rominf Thanks for your kind help :)

ircecho

@ircecho_gitlab

Can I ask what distance metric you used for the SemEval-2017 Task 2: Multilingual and Cross-lingual Semantic Word Similarity task? (Also, if the source code is available online somewhere, I would be happy to find out the answer myself, but I did not find the source code either.) I am sorry if I missed this information somehow. I read the respective paper and the blog entries pertaining to this. I found that for the 2018 task you used the dot product on unit vectors, did you use the same for the 2017 task?

amirouche

@amirouche

Follow up on approximate string matching over a bigger than memory dataset https://hyper.dev/blog/fuzzbuzz.html

Radhika Nikam

@radsn23

is using conceptnet for image recognition a good idea?

amirouche

@amirouche

@radsn23 I don't know.

amirouche

@amirouche

The prototype seems to work, here is the code: https://github.com/amirouche/fuzzbuzz/blob/master/fuzz.py#L126

Kaiqiang Huang

@kaiqiangh

@radsn23 Hi, I was considering a similar idea as you mentioned. To my knowledge, in ConceptNet, there might only contain concepts (image classes or object classes) and relationships between them. However, for image recognition task (I assume that you are talking about that simple task: one label for one image and given a new image, the model can predict its class label.), the model must know something from images itself such as image features. If you only consider using ConceptNet to solve this task, it seems like impossible unless you use additional image features with conceptNet. I am working on zero-shot learning for image and video classification where a large-scale knowledge graph might help it.

amirouche

@amirouche

Here is the detail of the locality sensitive hash: https://hyper.dev/blog/fuzzbuzz-hash-algorithm.html

Yondu Tsai

@yondu22

anyone have a way to translate all the english data to chinese? the chinese data is not as complete as english

amirouche

@amirouche

@yondu22 you can try to retrieve the wiktionary page and follow the chinse language link from there.

Yondu Tsai

@yondu22

Anybody have experience in comparing conceptnet on postgres vs neo4j...what are the advantages of neo4j, and which is more efficient for using conceptnet with same type of queries as the web api

Where communities thrive

People

Repo info

Activity