commonsense/conceptnet5

krgallagher

@krgallagher

The API is down for me as well

Bancherd

@Bancherd-DeLong

Hi, a newbie here. I tried to run "snakemake data/vectors/mini.h5" and received this error: File "/home/bancherd/.local/lib/python3.8/site-packages/wordfreq/tokens.py", line 264, in tokenize
tokens = _mecab_tokenize(text, language.language)
File "/home/bancherd/.local/lib/python3.8/site-packages/wordfreq/mecab.py", line 40, in mecab_tokenize
MECAB_ANALYZERS[lang] = make_mecab_analyzer(lang)
File "/home/bancherd/.local/lib/python3.8/site-packages/wordfreq/mecab.py", line 20, in make_mecab_analyzer
import ipadic
ModuleNotFoundError: No module named 'ipadic'
[Sun Aug 22 17:01:09 2021]
Error in rule miniaturize:
jobid: 0
output: data/vectors/mini.h5
shell:
cn5-vectors miniaturize data/vectors/numberbatch-biased.h5 data/vectors/w2v-google-news.h5 data/vectors/mini.h5
(exited with non-zero exit code)
I tried to look for "ipadic", without success. Can anyone suggest solutions? Thank you!

Bancherd

@Bancherd-DeLong

Hi, a newbie here. I tried to run "snakemake data/vectors/mini.h5" and received this error: File "/home/bancherd/.local/lib/python3.8/site-packages/wordfreq/tokens.py", line 264, in tokenize
tokens = _mecab_tokenize(text, language.language)
File "/home/bancherd/.local/lib/python3.8/site-packages/wordfreq/mecab.py", line 40, in mecab_tokenize
MECAB_ANALYZERS[lang] = make_mecab_analyzer(lang)
File "/home/bancherd/.local/lib/python3.8/site-packages/wordfreq/mecab.py", line 20, in make_mecab_analyzer
import ipadic
ModuleNotFoundError: No module named 'ipadic'
[Sun Aug 22 17:01:09 2021]
Error in rule miniaturize:
jobid: 0
output: data/vectors/mini.h5
shell:
cn5-vectors miniaturize data/vectors/numberbatch-biased.h5 data/vectors/w2v-google-news.h5 data/vectors/mini.h5
(exited with non-zero exit code)
I tried to look for "ipadic", without success. Can anyone suggest solutions? Thank you!

Inspite of the warning in pypi, I went ahead , installed "ipadic" and rerun the script: got the following(different) error:Building prefix dict from /home/bancherd/.local/lib/python3.8/site-packages/wordfreq/data/jieba_zh.txt ...
Dumping model to file cache /tmp/jieba.u600b79f75cbc9b33aa477293be70c0e2.cache
Loading model cost 0.057 seconds.
Prefix dict has been built successfully.
/usr/bin/bash: line 1: 37532 Killed cn5-vectors miniaturize data/vectors/numberbatch-biased.h5 data/vectors/w2v-google-news.h5 data/vectors/mini.h5
[Sun Aug 22 20:16:41 2021]
Error in rule miniaturize:
jobid: 0
output: data/vectors/mini.h5
shell:

Bancherd

@Bancherd-DeLong

Problem solved . I had too many opened chrome-tabs(even with 32G memory/ubuntu 20.04)

Evelyne

@echevry_gitlab

Hello, I am reviving an old project. I am trying to get the association through a list. In the past, the query was this: https://api.conceptnet.io/assoc/list/en/toast,cereal,juice@0.5,egg. does this feature is still available? If yes.. Where can I find the documentation? This query does not work anymore with the current API...

Gwen Rehrig

@dr-gwen

Hi all, I'm also building ConceptNet5 for the first time on a machine running Ubuntu 20.04 with 32 GB of RAM. I was able to run ./build.sh without any obvious errors that I saw in the output, but pytest is returning failed and skipped tests. Specifically:
test_languages.py fails (316), error message indicates it is unable to find the language_data module (traced to line 809 in .../langcodes/init.py).
test_json_ld.py fails as well with a KeyError on line 82 (which is: "quiz = ld[api('/c/en/quiz')]") and line 161 ("rel = ld[vocab('rel')])

Do these errors indicate that the installation was not successful and I should re-install? Or, have others encountered the same issues and have solutions? I did check the documentation and Googled the errors, but did not find any relevant troubleshooting solutions. Any suggestions would be appreciated.

3 replies

Bancherd

@Bancherd-DeLong

Hmm, could someone please tell me how to generate "numberbatch.txt.gz" file? It is NOT simply specifying in the snakemake file, is it?

1 reply

Carlos F. Enguix

@cenguix

Hi folks, I am currently writing a research survey paper about Open Knowledge Graphs. I am including Conceptnet. I would appreciate indeed any link/info indicating repository size and related fresh stats. I look forward to hearing from you. Best regards, Carlos F. Enguix

Sergey

@zababurinsv

Hi i want build conceptnet node.

But when i run build.sh i get this error

Error in rule convert_opensubtitles_ft:
    jobid: 0
    output: data/vectors/fasttext-opensubtitles.h5

RuleException:
CalledProcessError in line 663 of /home/zb/Desktop/conceptnet/Snakefile:
Command 'set -euo pipefail;  CONCEPTNET_DATA=data cn5-vectors convert_fasttext -n 2000000 data/raw/vectors/ft-opensubtitles.vec.gz data/vectors/fasttext-opensubtitles.h5' returned non-zero exit status 137.
  File "/home/zb/Desktop/conceptnet/Snakefile", line 663, in __rule_convert_opensubtitles_ft
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
Exiting because a job execution failed. Look above for error message
[Sat Sep 11 19:11:39 2021]
Finished job 206.
371 of 472 steps (79%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/zb/Desktop/conceptnet/.snakemake/log/2021-09-11T182112.270477.snakemake.log

What could be the reason for this?

1 reply

Sergey

@zababurinsv

How can I proceed with the installation instead of starting over?

I have.
300 GB of free disk space
At least 30 GB of available RAM
The time and bandwidth to download 24 GB of raw data

I start build.sh and on
464 of 472 steps (98%) done
I getting error.

/usr/bin/bash: line 1: 22394 Killed                  cn5-vectors intersect data/vectors/crawl-300d-2M-retrofit.h5 data/vectors/w2v-google-news-retrofit.h5 data/vectors/glove12-840B-retrofit.h5 data/vectors/fasttext-opensubtitles-retrofit.h5 data/vectors/numberbatch-retrofitted.h5 data/vectors/intersection-projection.h5
[Mon Sep 13 03:26:07 2021]
Error in rule merge_intersect:
    jobid: 177
    output: data/vectors/numberbatch-retrofitted.h5, data/vectors/intersection-projection.h5
    shell:
        cn5-vectors intersect data/vectors/crawl-300d-2M-retrofit.h5 data/vectors/w2v-google-news-retrofit.h5 data/vectors/glove12-840B-retrofit.h5 data/vectors/fasttext-opensubtitles-retrofit.h5 data/vectors/numberbatch-retrofitted.h5 data/vectors/intersection-projection.h5
        (exited with non-zero exit code)

Removing temporary output file data/psql/edges_gin.csv.
[Mon Sep 13 03:27:40 2021]
Finished job 3.
464 of 472 steps (98%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: /home/zb/Desktop/conceptnet5/.snakemake/log/2021-09-12T221420.189372.snakemake.log

2 replies

Sergey

@zababurinsv

After 6 hours of installation, I got an error.
Please tell me if it is possible to continue the installation after a failure?

Bancherd

@Bancherd-DeLong

It continues from just before the error, so should restart at step ~465.

jon.reeve

@jon.reeve:matrix.org

[m]

Does anyone know of a data source that can tell you the approximate or average dimensions of things? For example, I'd love to see something like "chair" also say "has an average height of about 1m, and an average width of about 0.5m."

lansheng

@huiguo07

I want to know whether the npy file in https://csr.s3-us-west-1.amazonaws.com/tzw.ent.npy uses one-dimensional embedding and glove to initialize some English entities? How is it built, if I want to build a similar npy file, how should I do it?

Ilya Lasy

@Misterion777

Hello everyone! Is there any implementation of concept extracting from sentence based on Conceptnet API, or I have to write it from scratch ? :D

4 replies

Gwen Rehrig

@dr-gwen

Hi all, I'm getting measures of semantic similarity between two English words from ConceptNet for a project I'm working on, and the values I get from the ConceptNet API (using the 'rel' query, e.g., https://api.conceptnet.io/relatedness?node1=/c/en/invaluable&node2=/c/en/unvaluable) differ from those I get from the raw Numberbatch embeddings (with no further tuning) by loading the data in word2vec format via gensim.models.KeyedVectors (per the example here: https://www.kaggle.com/danofer/poetry2vec-word-embeddings) and the .wv.similarity method. Here's an example of the code I'm using:

from gensim.models import KeyedVectors
model = KeyedVectors.load_word2vec_format('numberbatch-en-19.08.txt.gz',binary=False,unicode_errors='ignore',limit=800000)
model.wv.similarity("invaluable","unvaluable")

In some cases the difference is clearly just a matter of rounding (the API queries round to maybe 3 decimal places, in others it's more substantial. In the example pair "invaluable" and "unvaluable", ConceptNet's API gives me a relatedness of 0.455, and model.wv.similarity returns 0.251. I would get all of the comparisons from just one source and call it a day, but unfortunately it seems some of the words I'm comparing are not accessible via the API (or, at least, I haven't discovered a way to access them: for example, words or phrases that contain apostrophes).

Are the underlying data different between these two sources? Are the functions not equivalent? I'm new to word2vec so perhaps I'm using that wrong. Any advice would be appreciated. Thanks!

fuzzylemma

@fuzzylemma

Hey, is there an online api that allows users to use the random query function ? https://github.com/commonsense/conceptnet5/blob/master/conceptnet5/db/query.py#L200

razcafe

@razcafe

Hey, just trying to wrap my head around this. Is this an extension of the OpenCyc project or is does this have a different goal and happens to use OpenCyc as a foundation?

Zhanwen Chen

@zhanwenchen

Hey guys. The web app is throwing 500s now: https://conceptnet.io/c/en/chair

Zhanwen Chen

@zhanwenchen

Btw, I managed to build and deploy successfully on my end. If anyone needs help just ping me

5 replies

gautamsingh24

@gautamsingh24:matrix.org

[m]

I'm using this API to build the MCQ. As per my requirement I created the URL as "http://api.conceptnet.io/query?node=/c/en/%s/n&rel=/r/PartOf&start=/c/en/%s&limit=5" but it's throwing 500 error. Can anyone confirm is it working on your system or not? because Just want to confirm is this URL is correct or not.

1 reply

youmna

@Youmna-Salah

Hallo!

The app is down for sometime, any news about when it will be up and working again?

Zhanwen Chen

@zhanwenchen

Alternatively, there's a CSV for all the pre-built edges (assertions): https://github.com/commonsense/conceptnet5/wiki/Downloads. It's only 475MB

Zhanwen Chen

@zhanwenchen

Btw the CSV is tab separated without header. It's huge so I suggest using pandas like df = pd.read_csv(csv_fname, delimiter='\t', header=None)

Holly

@Hollie7

Hello! I meet the same problem that the server is down. Could anyone please be so kind to tell what's going on & will the server be repaired in a few days? Because I am conducting an experiment for my school work and really hope to take ConceptNet as an important comparison benchmark. Thanks a lot!

Motoki Yatsu

@m-yatsu

Is public Web API under maintenance? I can see a concept through Web browsers, but JSON-LD API is not working.

It always turns 500 status code.

Motoki Yatsu

@m-yatsu

It's nice to see you back!

farrokhsiar

@farrokhsiar

I am new to using conceptnet. Is there predefined embedding that covers all the ConceptNet entries? I tried Numberbatch, but it seems it does not cover all the entries.

Taewoon Kim

@tae898

Hello people!
Since the conceptnet server is often down, I made a simple docker installation setup that you can run locally.

Check out the repo: https://github.com/tae898/conceptnet5
Check out the video tutorial: https://youtu.be/UAM1XwbpOZI

Feel free to ask me questions.

Amirouche Amazigh BOUBEKKI

@amirouche

Hello people!
Since the conceptnet server is often down, I made a simple docker installation setup that you can run locally.
Check out the repo: https://github.com/tae898/conceptnet5
Check out the video tutorial: https://youtu.be/UAM1XwbpOZI
Feel free to ask me questions.

Thanks!

@tae898 by the way, I do not have the commit bit so I can not accept your pull-request.

Taewoon Kim

@tae898

@amirouche welcome! alright then is there someone else who can take care of the PRs?

Amirouche Amazigh BOUBEKKI

@amirouche

apparantly they are offline.

J Boyan

@jboyan

I'm curious why some common proper nouns like 'bezos' are missing from the word vector embeddings and conceptnet altogether? I thought dbpedia was an input source. https://conceptnet.io/c/en/bezos

Rohit.K

@RhtK07

Hi ! I want to enrich the audioset ontology with the help of conceptnet? Can anybody help me/guide me how can i do that?

And if i can, what all tools that are available that will be useful for me!

Thankyou in advance

Yonatan Bitton

@yonatanbitton

Hello
Do someone know how can I calculate relatedness fast?
For example:
https://api.conceptnet.io/relatedness?node1=/c/en/cinderella&node2=/c/en/ariel/n/wp/disney_character
I currently use the python API:

conceptnet_api = f'{conceptlite_lite_api_url}/relatedness?node1={node1}&node2={node2}'
req = requests.get(conceptnet_api).json()

Amirouche Amazigh BOUBEKKI

@amirouche

@rspeer is conceptnet maintenance still on your agenda given you changed job ?

Ilia Sucholutsky

@ilia10000

Hi!
What's the proper strategy for combining multiple word vectors? For example, if I want to compare how similar two sets of words are, what's the right way to use CNNB to do this?

ROAM447

@ROAM447

Hi all! I'm new to conceptnet, and i was trying to find different types of filters to use for the api. I specifically need synonyms and a more difficult to explain one. On the normal website if you type for instance, hello, then it will give you a list of what hello is a type of- e.g. greeting, airline, greeting that people sometimes use - Is there a filter for this?

Ifty Mohammad Rezwan

@imr165_twitter

Very newbie question here. I am working in collecting edge word of labels in conceptnet.

Ignore my previous message please

Very newbie question here. I am working in collecting edge word of labels in conceptnet.

Ifty Mohammad Rezwan

@imr165_twitter

Very newbie question here. I am working in collecting edge word of labels in conceptnet
As I was saying while querying, I manage to get the edges for single words in the api like for example when I query "albatross" like this "https://api.conceptnet.io/c/en/albatross" I get a good result with all the edges. But when I query "black footed albatross" like "https://api.conceptnet.io/c/en/black%20footed%20albatross" or "https://api.conceptnet.io/c/en/black_footed_albatross". I get the result that it is not a node in conceptnet. But what's funny is when I query "black footed albatross" on concepnet.io as "https://conceptnet.io/c/en/black_footed_albatross", I get synonym edges. There's english synonyms present too. So, I wondering If I was something here? Is the api limited in it's support for conceptnet or Am I missing something here. Thank You.

Ifty Mohammad Rezwan

@imr165_twitter

I apologize it seems to be working now

AlexMikhalev

@AlexMikhalev

Hello all, conceptnet5 deployment scripts rely on docker-compose files but there are no docker-compose.yml in the current repository https://github.com/commonsense/conceptnet5/blob/master/scripts/conceptnet.service, how has anyone managed to re-deploy conceptnet on their own servers?

25 replies

pyrator

@pyrator

Hi getting 500 error at the moment with https://api.conceptnet.io/c/en/example
{
"@context": [
"http://api.conceptnet.io/ld/conceptnet5.7/context.ld.json"
],
"error": {
"details": "Internal server error",
"status": 500
},
"version": "5.8.1"
}

3 replies

Where communities thrive

People

Repo info

Activity