Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    rocas82
    @rocas82
    ./pgloader -v -d commands
    the load of data startsand run for about 2 minutes and stop with this output:
    ./pgloader -v -d commands
    2016-06-16T13:56:21.730000+02:00 DEBUG MySQL: sending query:
    select column_name, data_type
    from information_schema.columns
    where table_schema = 'natesdb_master' and table_name = 'department'
    order by ordinal_position
    2016-06-16T13:56:21.730000+02:00 DEBUG MySQL: sending query: SELECT idDEPARTMENT, name, description, id_parent_department, path, location, eqtAvailableDefaultValue, idORGANIZATION FROM department;
    2016-06-16T13:56:21.730000+02:00 DATA < #("1" "OBS" "" NIL "/1/" "/OBS/" "1" "1")
    2016-06-16T13:56:21.730000+02:00 DATA < #("2" "Equant" "" "1" "/1/2/" "/OBS/Equant/" "1" "1")
    2016-06-16T13:56:21.730000+02:00 DATA < #("3" "FT-SCE" "" "1" "/1/3/" "/OBS/FT-SCE/" "1" "1")
    2016-06-16T13:56:21.730000+02:00 DATA < #("4" "tests" "" "3" "/1/3/4/" "/OBS/FT-SCE/tests/" "1" "1")
    2016-06-16T13:56:21.731000+02:00 DATA < #("5" "testsIt" "" "4" "/1/3/4/5/" "/OBS/FT-SCE/tests/testsIt/" "1" "1")
    2016-06-16T13:56:21.731000+02:00 DATA < #("6" "testsIng" "" "4" "/1/3/4/6/" "/OBS/FT-SCE/tests/testsIng/" "1" "1")
    2016-06-16T13:56:21.731000+02:00 DATA < #("7" "THATEST00" "" NIL "/7/" "/THATEST00/" "1" "1")
    2016-06-16T13:56:21.732000+02:00 DATA < #("8" "THATEST10" "" "7" "/7/8/" "/THATEST00/THATEST10/" "1" "1")
    2016-06-16T13:56:21.732000+02:00 DATA < #("9" "THATEST11" "" "8" "/7/8/9/" "/THATEST00/THATEST10/THATEST11/" "1" "1")
    2016-06-16T13:56:21.732000+02:00 DEBUG finish-batch[3] 9 rows in 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Reader for department is done in 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Finished processing READER for "department" 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Transformer 3 in action for department_restriction!
    I don't know what is going wrong. the out of the execution is freezed without any error message
    Then when I used CTR+C to exit, I have:

    2016-06-16T13:56:21.732000+02:00 DEBUG Finished processing READER for "department" 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Transformer 3 in action for department_restriction!
    ^C2016-06-16T14:16:27.047000+02:00 FATAL We have a situation here.
    An unhandled error condition has been signalled:
    Interactive interrupt at #x322AAE5197.

    Date/time: 2016-06-16-14:16An unhandled error condition has been signalled:
    Interactive interrupt at #x322AAE5197.

    Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {10071F67B3}>
    0: ("bogus stack frame")
    1: ((FLET #:WITHOUT-INTERRUPTS-BODY-646 :IN SB-THREAD::%CONDITION-WAIT))
    2: (SB-THREAD:CONDITION-WAIT #<SB-THREAD:WAITQUEUE Anonymous condition variable {10140174E3}> #<SB-THREAD:MUTEX "Anonymous lock" (free)> :TIMEOUT NIL)
    3: (LPARALLEL.CONS-QUEUE:POP-CONS-QUEUE/NO-LOCK #<unavailable lambda li

    rocas82
    @rocas82
    I can share pgloadr.log file
    rocas82
    @rocas82
    ERROR found in pgloader.log
    2016-06-16T13:56:21.727000+02:00 ERROR Database error 42P01: relation "database_version_dev_history" does not exist
    2016-06-16T13:56:21.728000+02:00 ERROR Database error 42P01: relation "database_version_dev_history" does not exist
    2016-06-16T13:56:21.728000+02:00 ERROR A thread failed with error: Database error 42P01: relation "database_version_dev_history" does not exist
    2016-06-16T13:56:21.728000+02:00 INFO Transformer[3] for database_version_dev_history is done in 0.058000s
    When a thread failed with error, does pgloader execution stop ?
    rocas82
    @rocas82
    Hello Dimitri, I solved out this issue. Thanks
    database_version_dev_history table doesn't exist in my postgresql db
    Dimitri Fontaine
    @dimitri
    Hi ! Did you mention this table explicitly in your load command? Can you open an issue on github so that I can reproduce and fix the issue? As you're saying errors like that should end in a clean FATAL message
    Chico Venancio
    @chicocvenancio
    Hello, I'm trying to build from sources in Windows and I'm getting problems with the freetds dll (sybdb.dll)
    I tried building it, or using different versions to no avail.
    Chico Venancio
    @chicocvenancio
    since I won't be using mssql I also tried the patch suggested at #159 to remove it, but I still get the problem
    rocas82
    @rocas82
    Hello Dimitri, Sorry for the delay. This is my answer to your last question. The problem was that the "database_version_dev_history" table exists in my MySQL database but it doesn't exist in my PostgreSQL database. So I explicitly excluded this table from load command with "EXCLUDING TABLE NAMES MATCHING 'database_version_dev_history'" and it works fine. So for me there was no bug. Many thanks
    Dimitri Fontaine
    @dimitri
    thanks, my pleasure!
    rocas82
    @rocas82
    Hello Dimitri,
    During data migration from mysql to postgresql database. I am had the following error message:
    Database error 22001: value too long for type character varying(5000)
    2 CONTEXT: COPY mass_treat_logs, line 2860, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 1613 : termin en Erreur : Des ..."
    3 Database error 22001: value too long for type character varying(5000)
    4 CONTEXT: COPY mass_treat_logs, line 769, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 2208 : termin en Erreur : Des ..."
    5 Database error 22001: value too long for type character varying(5000)
    6 CONTEXT: COPY mass_treat_logs, line 586, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 2534 : termin en Erreur : Des ..."
    7 Database error 22001: value too long for type character varying(5000)
    8 CONTEXT: COPY mass_treat_logs, line 3023, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5123 : termin en Erreur : Des ..."
    9 Database error 22001: value too long for type character varying(5000)
    10 CONTEXT: COPY mass_treat_logs, line 13, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5136 : termin en Erreur : Des ..."
    11 Database error 22001: value too long for type character varying(5000)
    12 CONTEXT: COPY mass_treat_logs, line 90, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5226 : termin en Erreur : Des ..."
    13 Database error 22001: value too long for type character varying(5000)
    14 CONTEXT: COPY mass_treat_logs, line 272, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5498 : termin en Erreur : Des ..."
    15 Database error 22001: value too long for type character varying(5000)
    16 CONTEXT: COPY mass_treat_logs, line 50, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5548 : termin en Erreur : Des ..."
    17 Database error 22001: value too long for type character v 16:25
    in one of my tables there is a column " logText varchar(5000)". Is there any restriction on size of varchar with pgloader ? Thanks for your feedback.
    Dimitri Fontaine
    @dimitri
    please open an issue, the problem seems to be with PostgreSQL, so I need to know if the table definition has been ported over by pgloader of if it's filling-up a pre-existing target?
    rocas82
    @rocas82
    I opened the issue #469 : dimitri/pgloader#469
    Dimitri Fontaine
    @dimitri
    answered on the issue and closed it, thanks!
    rocas82
    @rocas82
    Hello Dimitri
    regarding the issue #469, I posted some additional questions in the ticket. Could you have a look please ?
    Tengfei Niu
    @spartucus

    @dimitri Hello Dimitri, from the site

    Building for the Windows™ Operating System is easy enough and the platform is fully supported.

    and I have no knowledges of Lisp, how do i start build pgloader on Windows?
    Andrew Graham-Yooll
    @andrewgy8
    The readme says "Build error". I see the .spec file was updated 8 days ago. Is this representative of the project or is the project fine and this warning is erroneous for now?
    Dimitri Fontaine
    @dimitri
    The build error is Travis integration, the .travis.yaml file endured bitrot when Travis changed things in their hosting solution months ago and I never had time to properly handle that situation. Help welcome here!
    Johannes Visintini
    @joker234
    hey there. I want to move a ruby on rails production database from sqlite to postgres. I think your tool might be good for this. My problem is, that rails have to create the database schema to work properly with the new database. Can I use pgloader to get only the data
    so in SQL i would only want to have just the INSERT INTO statements
    I used “WITH data only”, but that also changed my tables;
    the indexes moved away after using pgloader
    Johannes Visintini
    @joker234
    Ah. Is it correct, that my problem should be solved by dimitri/pgloader@1023577 resp. dimitri/pgloader#504
    Andrew Graham-Yooll
    @andrewgy8
    @dimitri Sorryfor the late reply. I would help with that issue, although Im not sure what the fix would be. Perhaps a rewriting of the yml file?
    I could try something out this weekend...
    Andrew Graham-Yooll
    @andrewgy8

    Anyways, I have a question. Im sucessfully porting over numerous sqlite dbs to postgres and pgloader is doing a great job! However, Im running into a couple of errors. One is that that some of my sqlite tables dont have a unique id which messes up the addition of foreign keys into the postgres db. The error:
    Database error 42830: there is no unique constraint matching given keys for referenced table "shop"

    So I ran this in my load file:

    AFTER LOAD DO 
        $$ ALTER TABLE shop$ ADD UNIQUE (id);$$
    However, it didnt seem to fire that query and add the unique to the id column.
    Any thoughts?
    Dimitri Fontaine
    @dimitri
    can you add the same thing in the SQLite database so that it's converted from sources in the right shape?
    the after load do clause will only run after everything that pgloader does internally, there's no other hooks to run at intermediary stages yet (well, apart from the alter schema and alter table pgloader clauses that impact the catalogs used to migrate)
    Andrew Graham-Yooll
    @andrewgy8
    Yeah, thats what I was going to go for if those hooks didnt fire until after the pgloader processes.
    Another issue Im running into is using the type int with extra auto_increment to serial. It does not seem to add that modification to the postgres tables from the sqlite3 tables that have autoincrement.
    Andrew Graham-Yooll
    @andrewgy8
    This is my current load file which doesnt seem to convert auto-incremnt to serial:
    
    LOAD DATABASE
         from ../../data/databases/main_db.sqlite
         into postgresql://postgres:postgres@127.0.0.1:5432/main_db
    
    WITH include drop, 
        create tables, 
        create indexes, 
        reset sequences
    
    CAST type datetime to timestamp drop default drop not null using zero-dates-to-null, 
    type int with extra auto_increment to serial keep typemod keep default keep not null 
    
    SET work_mem to '16MB', 
        maintenance_work_mem to '512 MB';
    Dimitri Fontaine
    @dimitri
    @andrewgy8 please see dimitri/pgloader#556, should be fixed now
    Andrew Graham-Yooll
    @andrewgy8
    Fantastic @dimitri !
    Ill look into the timezone issue. Time is such a strange thing...
    Marcel Hoppe
    @hobbypunk90

    Hello,
    i habe a little problem, i'm using the 3.1 version an a debian machine with this little script:

    LOAD DATABASE
         FROM    mysql://<user>:<password>@localhost/gildamesh_test_db
         INTO postgres:///gildamesh_test
     WITH data only, truncate;

    and i get this very strange error:
    Error on Github