OpenRefine/OpenRefine

Welcome!

Binita Kumari

@Binita-tech

@wetneb, Can I follow template given by @antoine2711 for your projects?

1 reply

shariffff

@shariffff:matrix.org

[m]

Hi @antoine2711 have commited some vital issue here and changes https://github.com/OpenRefine/OpenRefine/pull/4810/files perhaps we need more discussion about this idea to affirm clean code of conduct.

gitonthescene

@gitonthescene

FWIW, I'm seeing reconciliation-api/specs#82

Shriya

@shriyasankhyan

Whenever I am doing $('').html(showNonPrintableChars(cell.v)).appendTo(divContent);
, my code is failing due to cypress error.
I checked the tests on my local system and it pointed me towards
https://www.slf4j.org/codes.html#StaticLoggerBinder but still I am not able to understand in which direction I should work for fixing the parsing panel. I would be grateful if you help me with the same

It is in OpenRefine/OpenRefine#4798

Walton Goga

@WaltonG

@elroykanye Congratulations on your successful outreachy internship application.
@antoine2711 I am grateful and excited to be your mentee.
Am looking forward to a wonderful experience with the community during my internship period.

2 replies

Elroy Kanye

@elroykanye

Congrats to you, @WaltonG
I am glad to be here and contributing during and after this period

2 replies

asgreenb

@asgreenb

Hi folks, apologies if this is the wrong place to post this Q, but... I am wondering if OpenRefine is appropriate for the following use case.

Within our organization, we have multiple Clinical Practice Locations (places that a patient can go for clinical care). Approximately 600 locations/ practices.

We have ~7 computer systems which maintain their own lists / databases of these locations (name, ID, street 1-2, city, state, zip, phone, etc)

Can I

import each of these lists,
normalize the fields from each source (aka “Suite” in one list = “Street 2” in another) – and save this mapping,
match/ group them as the ‘same location’ to create a master location with 7 sub-records (or more - if there are duplicates in the source files),
for each filed in the master record, pick which of the 7+ values should be the ‘surviving’ one
generate a list for the owners of the source systems to update their information from X to Y.

Thank you
Adam

11 replies

Cece (Sixin Li)

@ceceli:matrix.org

[m]

Hi all, my pull request failed a CI/CD test. The error message is

failed to execute goal net.revelc.code.formatter:formatter-maven-plugin:2.17.1:validate (default-cli) on project wikidata

. I just tried to use the formatter-maven-plugin to format my file in IntelliJ but it requires some config file. I wonder what config file OpenRefine uses or what code format OR is following so that I can format manually.

Jan Chaloupecky

@JanC

Hi,
are there any known issues of workspaces not being accessible after closing / reopening OpenRefine? It seems like I have to re-create my workspace everytime I open OR. The Open Project tab is empty and does not show recent projects. I tried deleting completely my OR data folder but it did not help. I tried with both 3.5.2 and 3.5.1

Jan Chaloupecky

@JanC

I noticed that in the OpenRefine data folder, there are no subfolders created (such as 1742324759816.project ). It used to work fine though. I can also reproduce it quite consistently on two different computers

Jan Chaloupecky

@JanC

It looks like OpenRefine does not have the time to save the project when I close the Terminal. Could it be related to Windows 11? All works fine on macOS

John Muccigrosso

@Jmuccigr

Hi, I'm working on a dataset that's got a bunch of unicode characters that represent letters with "dot under". I'm wondering if there's a way to convert them to their simpler ASCII-like form. For example, ạ to a. I find a few utilities for Python (e.g., https://github.com/ajanin/uni2ascii ), but am hoping to keep this in OpenRefine.

For comparison another set of letters are underscored, but in those cases the combining underscore was used and that's susceptible to a simple gsub in R. TIA!

9 replies

Jan Chaloupecky

@JanC

hey, I've previously setup a development version for OpenRefine without too many issues. I'm now setting it up on a new mac. mvn compile and ./refine run without errors but opening the UI yields a lot of js errors.

I find out that I needed to npm install from the main/webapp folder first. I believe that should be part of the main readme
https://github.com/OpenRefine/OpenRefine#run-from-source

1 reply

Frederik Elwert

@elwerfkp:ruhr-uni-bochum.de

[m]

Hi! I’m using the reconciliation feature to map terms to Getty AAT. Sometimes it doesn’t find a good match, and I would like to try it with another term. Is it possible to re-run the reconciliation only for selected cells?

2 replies

regisrob

@regisrob:matrix.org

[m]

Hi all! I just saw the release of version 3.6.0 by @wetneb Would it be possible to integrate into this release the latest commits to language files?

particularly I am thinking of the modifications made to OpenRefine/extensions/wikidata/module/langs/translation-fr.json. See https://github.com/OpenRefine/OpenRefine/commits/master/extensions/wikidata/module/langs/translation-fr.json

the last two commits from June 21 and July 19 are not included in 3.6, is there a reason why it is not the case?

5 replies

regisrob

@regisrob:matrix.org

[m]

Ok I understand, that makes sense. Thanks for your answer

I was not aware those changes were made after the release candidate

Frazer-Nyambe

@Frazer-Nyambe

Hi Everyone..I am having a file that contains a lot of records in it and when I export these records as .XSL format, one record is split into multiple rows in an Excel file. I would like to join the rows for each particular record into one row Using OpenRefine, such that each row should contain one record's details.

8 replies

nharodig

@nharodig

Hello everyone, I have a question about the OpenRefine container, I need to alert in case the server fails or goes down, for that I was thinking of checking if the container is running, for this I would like to know if the container stops in case OpenRefine fails because if the server stops working and the container continues to run, it would not alert properly.

1 reply

Barn Buster

@BarnBuster_twitter

New user here. How do I capitalize the first letter of the first word of each individual entry within a cell on a TSV? For example is there a function to make 'Dogs||cats||fish' change to 'Dogs||Cats||Fish' and do the same to all cells on the TSV?

26 replies

Steve Maser

@stevemaser_gitlab

Are any of the OpenRefine devs here? 3.6.0 will not launch on the Mac with munki-deployed "root:wheel" permissions on the app (3.5 worked fine like that.) Might somebody be able to take a look to see if you could fix that? We can't otherwise deploy 3.6 here.

32 replies

Antonin Delpeuch

@wetneb

Hello all, tomorrow at 9 August 2022 14:00 UTC, we have a meetup for OpenRefine contributors to talk about anything, and you can join it with this link: https://us02web.zoom.us/j/84496232592?pwd=NHBpak1FODdDWDNMQWNENTllMzlKUT09

Anna Goslen

@anna_goslen_twitter

Hey all, I was wondering if there is a way in any of OpenRefine's export options to export one file per row. Essentially I have two columns, one with an id and one with text and I would like to create several text files (1 per row) named by the id in the id column. I've been figuring I would need to script this task outside of OpenRefine?

1 reply

teknowledgist

@teknowledgist

Can OpenRefine be "installed" to work for all (unprivileged) users of a (Windows)computer? Why does it need to write to its program directory? It's smart enough to make the workspace within the user profile, so is there a way to make all writes/edits fall within the user profile allowing the program directory to be in a read-only location?

1 reply

Antonin Delpeuch

@wetneb

Hi all! We have the OpenRefine contributor hang out now :) Feel free to join to talk about anything! https://us02web.zoom.us/j/84496232592?pwd=NHBpak1FODdDWDNMQWNENTllMzlKUT09

Sandra Fauconnier

@trnstlntk

Hi everyone! We are considering to move OpenRefine's mailing lists and Gitter (this space!) to a web-based Discourse forum. You can read more about the considerations here: https://openrefine.org/blog/2022/09/20/discourse.html and we welcome your feedback.

Felipe

@coluccini

Hi Everyone! Sorry in advance if this is not the place to ask this type of help and if this a too dumb question. I'm clustering a list using nearest neighbor to clean up duplicates. The thing is that I can't get 2 values (that I know that are there and that are pretty similar) to be clustered. The values are "react" and "react.js" and I'm not seeing clustered with any clustering setting I've tried. How can I do it?
PS: My worry is that if that two words are not getting clustered I might be missing some other similar words that I'm not aware of.

18 replies

JP Hansen

@jphansendk

I'm new to OpenRefine and RegEx - Im actually not into expressions at all.. A sample of my data in a line could be "iPhone XS 64GB Space Grey" or "Galaxy S21 5G 8GB+256GB Black" and i'm trying to isolate data in new columns with the cell data to "Model", "Size" and "Color". Can I isolate each values with expressions? Also is it possible to to correct/replace values in a cell from a list I have? Like if I have a list with all the correct colors and want to extract the cells containing the color names from my list to make the new column with the colors only? Hope someone can help me out :-) TIA

6 replies

Owen Stephens

@ostephens

15 replies

JP Hansen

@jphansendk

So is there an expression that could like this value.replace("cells.Size.value","") where the value from column "Size" is the value that needs to be deleted in the present column.

4 replies

Phil Vigus

@PhilipVigus

Error log

6 replies

Hey all, I'm trying to get OpenRefine set up locally with IntelliJ IDEA. I largely have it working, but when I run the tests I get a bunch of errors, even though the tests are passing. I'm attaching a copy of the error log as it's too long to fit into chat. I'd be extremely grateful if someone could point me in the right direction, as I'm sure it must be something I've missing during the setup.

Antonin Delpeuch

@wetneb

In three hours we have a contributor meetup! An hour to chat about any topic you would like, work together on something… It depends on who shows up! Details are here: https://groups.google.com/g/openrefine-dev/c/T53nTVWAi_o

Robert Garrigos

@robertgarrigos

Hi everyone, I’m new to Openrefine and have a problem I don’t know how to solve: I’m using openrefine to edit items on wikidata. I have a first list of items, which I reconcile against WD, then add a new column, with the values of "instance of”, from reconciled values. This leaves me with some items with one "instance of" value and some other with two. How can I add a second value for "instance of" in the records with only one? thanks!

Jan Chaloupecky

@JanC

hey,
are there any examples of extensions with UI which is opened as a standalone page instead of being contained in a dialog?
backgroud:

I wrote an extension where I expose a new Grel function and it's working fine. This Grel function uses a small yaml database and I would like to offer the user to add entries to that db via the UI.
For that purpose, I created a Command with basic rest CRUD operations and those work fine as well.
Now I'm implementing the actual UI and by mimicking other extensions I manage to add a entry to the main ui using ExtensionBar.MenuItems.push(
That menu opens a new dialog in the same way as the clustering dialog: https://github.com/OpenRefine/OpenRefine/blob/master/main/webapp/modules/core/scripts/dialogs/clustering-dialog.js#L156
However, the UI of my extensions is a bit more complex so a "dialog" is not enough to fit all the content and I'd like it to open in a standalone window/tab without any of the OpenRefine UI or menus.
How can I do that?

7 replies

abbe98

@abbe98:matrix.org

[m]

Have someone recently got this Geonames reconciliation endpoint to work? It seems to work in the test bench but not in OpenRefine: https://api.exldevnetwork.net/geonames-openrefine/reconcile The error being "Failed to guess cell types for load".

Andreas Wagner

@g-0435856:matrix-test.gwdg.de

[m]

abbe98: It seems that it does not support POST requests. See jweisman/geonames-openrefine#7 (I didn't find the time to contribute yet.)

abbe98

@abbe98:matrix.org

[m]

Is there an older version of OpenRefine which supports GET requests, or was it never compatible?

3 replies

Tom Morris

@tfmorris

It looks like this was switched from GET to POST in October 2020 for 3.5-beta, so perhaps 3.4 and earlier are unaffected https://github.com/OpenRefine/OpenRefine/commit/9ac54edbba2fd0876b3f2f99c8bc1588799ecb3b#diff-e0b9b0aa8abc01cf3d5c686dbdc5fb6e6ad874ee35bc5d699308987615adb8daL173-L191

Davinder Ratwal

@Dave8289

helo

abbe98

@abbe98:matrix.org

[m]

Thanks @tfmorris I will give 3.4 a try!

tifo

@tifo:matrix.org

[m]

I am hoping for someone to point me in the right direction. I have setup Eclipse but when I go to build/run I get an error saying that a plug-in type cannot be found for indentlayer and console. I have tried setting up IntelliJ, when I try to build I get an error when it is building packages and the error says it cannot perform a chmod (I’m on Windows). I am sure these are really simple fixes in configuration of the IDE but I’m at a loss for where to look for guidance. Any help greatly appreciated.

4 replies

Lobaluna

@lobaluna:matrix.cuates.net

[m]

I guess you are building from the sources. I'm sorry to not be able to help you. Good luck

Andreas Wagner

@g-04358561:matrix-test.gwdg.de

[m]

Hi all, in order to clean data that is distributed over several tables in a relational db, I import them with a long SQL SELECT statement that joins all the tables. Now I have what would be nested records (in concrete terms, several territories that each have several legislators who in turn have passed several laws that each finally concern several subject matters). Is there (documentation for) a way to handle such nested records?

11 replies

Andreas Wagner

@g-04358561:matrix-test.gwdg.de

[m]

:point_up: Edit: Hi all, in order to clean data that is distributed over several tables in a relational db, I import them with a long SQL SELECT statement that joins all the tables. Now I have what would be nested records (in concrete terms, several territories that each have several legislators who in turn have passed several laws that each finally concern several subject matters). Is there (documentation beyond this manual page for) a way to handle such nested records?

Bhaswati Roy

@BhaswatiRoy

Hello everyone I am Bhaswati Roy, a prefinal year CSE student interested in Machine Learning
I was going through the openrefine.org repository and would love to work on the following issue
OpenRefine/openrefine.org#153 (the phrase could direct to contributing.md file so that contributors can get started)
It would be great if I could be assigned (I have also commented below the issue)

Bhaswati Roy

@BhaswatiRoy

Hello everyone
I wanted to work on this issue, and have left my idea in the comment section, some reviews on the idea would be appreciated!
OpenRefine/openrefine.org#54

2 replies

Where communities thrive

People

Repo info

Activity