Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Dimitri Fontaine
    @dimitri
    @mcapanema : can you detail your JSON use case a bit please?
    sspr28
    @sspr28
    Thank you Dimitri...er are tryong to load from Text file.
    sorry for typo...We are trying to load from a text file
    sspr28
    @sspr28
    Hi All,
    I want to skip the 1st line in pgloader.configuration file..I have checked using skip head_lines=1 in my configuration but am getting the error global name 'nb_lines' is not defined..
    Could you help on this .
    rocas82
    @rocas82
    Hi All,
    I would like to build a pgloader binary on my centOS 6.7 from source code:
    rocas82
    @rocas82
    first I used bootstrap-centos.sh to create the environment, then run make and I have the following error message: debugger invoked on a LOAD-SYSTEM-DEFINITION-ERROR in thread #<THREAD "main thread" RUNNING {1002A8BA13}>: Error while trying to load definition for system bordeaux-threads from pathname /home/pgloader-3.2.2/build/quicklisp/dists/quicklisp/software/bordeaux-threads-v0.8.5/bordeaux-threads.asd: READ error during COMPILE-FILE: You need ASDF >= 3.1 to load this system correctly.(in form starting at line: 1, column: 0, file-position: 0)
    could you help on that please ?
    rocas82
    @rocas82
    Could you please help me to compile pgloader-3.2.2 on CentOS 6.7 to obtain a binary format ?
    is there any procedure that I can follow to compile pgloader-3.2.2 on Centos 6.7 ?
    Thank sfor your help
    Dimitri Fontaine
    @dimitri
    The ASDF being too old often is because the compiler itself (SBCL) is too old, which version do you have?
    rocas82
    @rocas82
    I had got the issue with sbcl-1.1.14 , then I installed SBCL 1.2.6 and it seems better,
    I loaded pgloader "3.2.02ecd88" binary format as it is available on your github and I am trying to use it
    Do I need to install SBCL before runing this pgloader binary format ?
    Dimitri Fontaine
    @dimitri
    If you download a pgloader binary, it's self contained (it includes SBCL) so you don't need SBCL at run-time. However it's a good idea to fetch the current git master's version and build from sources, and for that you need SBCL installed.
    rocas82
    @rocas82
    Hello Dimitri

    ./pgloader -V

    pgloader version "3.3.0.50"
    compiled with SBCL 1.3.5
    When I run pgloader, I have got the following error:
    mkdir /tmp/pgloader
    [root@VM_kurmi_Perso pgloaderAndTools-6.2.0]# ./pgloader -v commands
    2016-06-16T12:23:53.269000+02:00 NOTICE Starting pgloader, log system is ready.
    2016-06-16T12:23:53.569000+02:00 LOG Main logs in '/tmp/pgloader/pgloader.log'
    2016-06-16T12:23:53.624000+02:00 LOG Data errors in '/tmp/pgloader/'
    2016-06-16T12:23:53.624000+02:00 LOG Parsing commands from file #P"/root/pgloaderAndTools-6.2.0/commands"
    An unhandled error condition has been signalled:
    PostgreSQL protocol error: Unexpected message received: Z
    rocas82
    @rocas82
    When I run pgaloader in debug mode as following:
    ./pgloader -v -d commands
    the load of data startsand run for about 2 minutes and stop with this output:
    ./pgloader -v -d commands
    2016-06-16T13:56:21.730000+02:00 DEBUG MySQL: sending query:
    select column_name, data_type
    from information_schema.columns
    where table_schema = 'natesdb_master' and table_name = 'department'
    order by ordinal_position
    2016-06-16T13:56:21.730000+02:00 DEBUG MySQL: sending query: SELECT idDEPARTMENT, name, description, id_parent_department, path, location, eqtAvailableDefaultValue, idORGANIZATION FROM department;
    2016-06-16T13:56:21.730000+02:00 DATA < #("1" "OBS" "" NIL "/1/" "/OBS/" "1" "1")
    2016-06-16T13:56:21.730000+02:00 DATA < #("2" "Equant" "" "1" "/1/2/" "/OBS/Equant/" "1" "1")
    2016-06-16T13:56:21.730000+02:00 DATA < #("3" "FT-SCE" "" "1" "/1/3/" "/OBS/FT-SCE/" "1" "1")
    2016-06-16T13:56:21.730000+02:00 DATA < #("4" "tests" "" "3" "/1/3/4/" "/OBS/FT-SCE/tests/" "1" "1")
    2016-06-16T13:56:21.731000+02:00 DATA < #("5" "testsIt" "" "4" "/1/3/4/5/" "/OBS/FT-SCE/tests/testsIt/" "1" "1")
    2016-06-16T13:56:21.731000+02:00 DATA < #("6" "testsIng" "" "4" "/1/3/4/6/" "/OBS/FT-SCE/tests/testsIng/" "1" "1")
    2016-06-16T13:56:21.731000+02:00 DATA < #("7" "THATEST00" "" NIL "/7/" "/THATEST00/" "1" "1")
    2016-06-16T13:56:21.732000+02:00 DATA < #("8" "THATEST10" "" "7" "/7/8/" "/THATEST00/THATEST10/" "1" "1")
    2016-06-16T13:56:21.732000+02:00 DATA < #("9" "THATEST11" "" "8" "/7/8/9/" "/THATEST00/THATEST10/THATEST11/" "1" "1")
    2016-06-16T13:56:21.732000+02:00 DEBUG finish-batch[3] 9 rows in 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Reader for department is done in 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Finished processing READER for "department" 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Transformer 3 in action for department_restriction!
    I don't know what is going wrong. the out of the execution is freezed without any error message
    Then when I used CTR+C to exit, I have:

    2016-06-16T13:56:21.732000+02:00 DEBUG Finished processing READER for "department" 0.018000s
    2016-06-16T13:56:21.732000+02:00 DEBUG Transformer 3 in action for department_restriction!
    ^C2016-06-16T14:16:27.047000+02:00 FATAL We have a situation here.
    An unhandled error condition has been signalled:
    Interactive interrupt at #x322AAE5197.

    Date/time: 2016-06-16-14:16An unhandled error condition has been signalled:
    Interactive interrupt at #x322AAE5197.

    Backtrace for: #<SB-THREAD:THREAD "main thread" RUNNING {10071F67B3}>
    0: ("bogus stack frame")
    1: ((FLET #:WITHOUT-INTERRUPTS-BODY-646 :IN SB-THREAD::%CONDITION-WAIT))
    2: (SB-THREAD:CONDITION-WAIT #<SB-THREAD:WAITQUEUE Anonymous condition variable {10140174E3}> #<SB-THREAD:MUTEX "Anonymous lock" (free)> :TIMEOUT NIL)
    3: (LPARALLEL.CONS-QUEUE:POP-CONS-QUEUE/NO-LOCK #<unavailable lambda li

    rocas82
    @rocas82
    I can share pgloadr.log file
    rocas82
    @rocas82
    ERROR found in pgloader.log
    2016-06-16T13:56:21.727000+02:00 ERROR Database error 42P01: relation "database_version_dev_history" does not exist
    2016-06-16T13:56:21.728000+02:00 ERROR Database error 42P01: relation "database_version_dev_history" does not exist
    2016-06-16T13:56:21.728000+02:00 ERROR A thread failed with error: Database error 42P01: relation "database_version_dev_history" does not exist
    2016-06-16T13:56:21.728000+02:00 INFO Transformer[3] for database_version_dev_history is done in 0.058000s
    When a thread failed with error, does pgloader execution stop ?
    rocas82
    @rocas82
    Hello Dimitri, I solved out this issue. Thanks
    database_version_dev_history table doesn't exist in my postgresql db
    Dimitri Fontaine
    @dimitri
    Hi ! Did you mention this table explicitly in your load command? Can you open an issue on github so that I can reproduce and fix the issue? As you're saying errors like that should end in a clean FATAL message
    Chico Venancio
    @chicocvenancio
    Hello, I'm trying to build from sources in Windows and I'm getting problems with the freetds dll (sybdb.dll)
    I tried building it, or using different versions to no avail.
    Chico Venancio
    @chicocvenancio
    since I won't be using mssql I also tried the patch suggested at #159 to remove it, but I still get the problem
    rocas82
    @rocas82
    Hello Dimitri, Sorry for the delay. This is my answer to your last question. The problem was that the "database_version_dev_history" table exists in my MySQL database but it doesn't exist in my PostgreSQL database. So I explicitly excluded this table from load command with "EXCLUDING TABLE NAMES MATCHING 'database_version_dev_history'" and it works fine. So for me there was no bug. Many thanks
    Dimitri Fontaine
    @dimitri
    thanks, my pleasure!
    rocas82
    @rocas82
    Hello Dimitri,
    During data migration from mysql to postgresql database. I am had the following error message:
    Database error 22001: value too long for type character varying(5000)
    2 CONTEXT: COPY mass_treat_logs, line 2860, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 1613 : termin en Erreur : Des ..."
    3 Database error 22001: value too long for type character varying(5000)
    4 CONTEXT: COPY mass_treat_logs, line 769, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 2208 : termin en Erreur : Des ..."
    5 Database error 22001: value too long for type character varying(5000)
    6 CONTEXT: COPY mass_treat_logs, line 586, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 2534 : termin en Erreur : Des ..."
    7 Database error 22001: value too long for type character varying(5000)
    8 CONTEXT: COPY mass_treat_logs, line 3023, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5123 : termin en Erreur : Des ..."
    9 Database error 22001: value too long for type character varying(5000)
    10 CONTEXT: COPY mass_treat_logs, line 13, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5136 : termin en Erreur : Des ..."
    11 Database error 22001: value too long for type character varying(5000)
    12 CONTEXT: COPY mass_treat_logs, line 90, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5226 : termin en Erreur : Des ..."
    13 Database error 22001: value too long for type character varying(5000)
    14 CONTEXT: COPY mass_treat_logs, line 272, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5498 : termin en Erreur : Des ..."
    15 Database error 22001: value too long for type character varying(5000)
    16 CONTEXT: COPY mass_treat_logs, line 50, column logtext: "Opration UP1TR023-60707-1397462564755 : Quick Feature : lment 5548 : termin en Erreur : Des ..."
    17 Database error 22001: value too long for type character v 16:25
    in one of my tables there is a column " logText varchar(5000)". Is there any restriction on size of varchar with pgloader ? Thanks for your feedback.
    Dimitri Fontaine
    @dimitri
    please open an issue, the problem seems to be with PostgreSQL, so I need to know if the table definition has been ported over by pgloader of if it's filling-up a pre-existing target?
    rocas82
    @rocas82
    I opened the issue #469 : dimitri/pgloader#469
    Dimitri Fontaine
    @dimitri
    answered on the issue and closed it, thanks!
    rocas82
    @rocas82
    Hello Dimitri