Where communities thrive

  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
Repo info
    was the issue fixed? I have 3.6.1
    John McCarthy
    Im running dimitri/pgloader:latest and trying to move a mssql db to a postgres db
    I've can't seem to get it to work. I'm currently stuck at this error: FATAL error: Undefined alien: "SSLeay"

    pgload file contents:

    load database
         from mssql://SA@
         into postgresql://postgres:'P@@ssword'@localhost/serverautomation
    before load do $$ drop schema if exists dbo cascade; $$;

    pgloader command: pgloader myfile --verbose --debug

    Can pgloader convert SQL Server .sql files to PostgreSQL sql files?
    like table definitions etc
    or is it only for running databases
    I'm getting errors trying to run pgloader to convert a .sql file
    2019-06-20T13:37:08.008000-07:00 LOG pgloader version "3.6.1"
    FATAL error: At
      -- ********************
      ^ (Line 1, Column 0, Position 0)
    In context KW-LOAD:
    While parsing KW-LOAD. Expected:
         the character Tab
      or the character Newline
      or the character Return
      or the character Space
      or the string "--"
      or the string "/*"
      or the string "load"
    clearly -- is one of the expected strings
    yet it explodes
    Ben Chiciudean
    I'm loading from csv file and importing to postgresql, how can I define default boolean value for column in target table?

    will this work:

    meta_type,state,is_imported boolean using TRUE

    this is on target columns

    Luke Snape
    Hey dimitri, having trouble building the pgloader:debian docker image: Undefined foreign library: CL+SSL::LIBEAY32 when trying to run the container suggests to me it's a Debian dependency issue?
    You're using stable-slim, not a pinned version. However, I tried stretch-slim and jessie-slim and it fails for different reasons.
    Luke Snape

    @dimitri not debian package versions. We think this is the culprit: cl-plus-ssl/cl-plus-ssl@f67a488

    The cl+ssl fix is not yet available via quicklisp. Any way to work around this problem in the meantime?

    Alejandro Matos
    I installed pgloader from apt, when I run it it doesn't show any errors, it finishes but no data has migrated to postgres, how could I debug this?
    Varun Dutta
    Hi anyone tried passing cast rule directly in docker run command like docker run --security-opt seccomp=unconfined --rm --name pgloader --net=host dimitri/pgloader:latest pgloader mysql://xxxx:3306/abc pgsql://xxxx:5432/abc --cast 'type bigint to int8 drop typemod' .. i tried passing all the combinations like --cast just after pgloader, in double quotes .. it is not working giving errors
    Dimitri Fontaine
    Hi guys, thanks all for your interest and questions here, I'm not sure I can handle all of them, though let's try.
    About SSL, it all depends on the OS and the packaging, and Common Lisp does not make that parts as easy as I would like. I must admit I'm sometimes lots in what needs to be done for the whole thing to be happy... I know that we solved it in the Debian packaging.
    About loading data to and from SQL files, pgloader does NOT do that.
    About defining a default value for a target column, either use the CAST clause of the pgloader's command, or define your target table yourself in the target database or in the BEFORE LOAD commands.
    Hi Everyone, just wondering if anyone is still running into "SYB-MSDATETIME2" errors, I've been though the number of earlier github issues around this, running on the latest 3.6.1 and can see the usual recommendations are all applied there.
    Well, actually the errors change between versions. On the latest from Master, getting SSL errors "OpenSSL_version_num" this is the same between Github src, and the docker images.
    Brian McMahon
    First happy holidays to all! I wanted to ask if anybody else has had an issue with keeping defaults values from the original DB? I've tried casting multiple times and it doesn't get built into the DDL in the destination DB? Has anybody else had issues keeping the default value in the DDL transferred to the destination DB?
    hi, we want to load 'bytea' type table with hex string, however, instead of inserting 9116452417 we get 3931313645324137 . would u know? (we are migrating from oracle to postgresql and 'raw' data type was suggested to convert to 'bytea', should we define the bytea column differently?)
    we have tried to alter the database to have bytea_output set to 'hex' but we couldn't overcome the above without adding '\x<hex[string>', is there a way to add '\x' fixed into the pgloader script?
    @dimitri I commented on #998 on Github, which possibly solves a few SSL errors I had, wondering your thoughts on adding the extra dev libs to the build?
    Joe Koberg
    Hello folks. Seems like I can't get pgloader to finish without hanging at the end of the process
    I am copying a mysql DB to a local postgres DB, and everything seems to go great, but then all CPU and network activity stops and nothing seems to happen
    I can't seem to get a clean run without some error like the following popping up, but it proceeds after that point so I assume it retries
    debugger invoked on a PGLOADER.CONNECTION:DB-CONNECTION-ERROR in thread    
    #<THREAD "lparallel" RUNNING {10064C0FD3}>:
      Failed to connect to mysql at ... The condition Socket error in "connect": EINTR (Interrupted system call) occurred with errno: 0.
    The current thread is not at the foreground,
    SB-THREAD:RELEASE-FOREGROUND has to be called in #<SB-THREAD:THREAD "main thread" RUNNING {10005284C3}>
    for this thread to enter the debugger.
    Joe Koberg
    well, this time it went through. thanks anyway!
    Jérémy Leherpeur
    Is it possible to install 3.6.1 version on ubuntu without compile ?
    Christoph Berg
    @amenophis: 3.6.1 builds on eoan only, you can install it from apt.postgresql.org (and on focal once we start supporting that)
    Jérémy Leherpeur
    @df7cb thanks
    I compiled it with a patch for PG12 on each machine which needs pgloader ;)
    Jérémy Leherpeur

    I tried to run this command

    ./pgloader/build/bin/pgloader -v \
    --context ./pgloader.ini \
    --summmary ./summary.log \
    file1.load \
    file2.load \

    The issue is the summary.log file contains only summary for file3.
    Did I miss something ?

    Emre Küçük
    sb-impl::default-external-format :UTF-8
    tmpdir: #P"/tmp/pgloader/"
    2020-02-28T14:08:08.031000Z NOTICE Starting pgloader, log system is ready.
    2020-02-28T14:08:08.055000Z INFO Starting monitor
    2020-02-28T14:08:08.063000Z INFO Stopping monitor
    i am executing the pgloader command and there is no table created in postgresql database
    and i get the above logs
    I am converting legacy database (dBASE III) with encoding IBM (german) using :

    FROM 'daten.dbf' WITH ENCODING CP850
    INTO postgresql:///staging
    WITH truncate, create table

    and it works well with only a small glitch. Beside the table 'public.daten', I also got a schema 'daten' with no dependencies at all.

    has the --config option been deprecated?
    I am trying to use a .load but every time i ran pgloader csv-writer.load I get an error saying: No such file or directory
    I am using pgloader version "3.6.2"
    we are having an issue where we have some images stored in a database that we're moving from mssql into pg and the fields storing the image data seems to be truncating at 4096 - any ideas if there is an option we need to set to allow larger chunks of data to pass through?
    1 reply
    Hello everyone and hello @dimitri
    I have a question about pgloader. I want to import a table from MSSQL database to a PGSQL.
    I use pgloader version 3.6.3~devel from docker image dimitri/pgloader:latest with a modified freetds conf explained in pgloader readthedoc
    I launch pgloader like this
    According to debug log, it create the table, read the source data but imported nothing
    In the log i found
    Finished processing WRITER for "dbo.csection" 31.060000s
    writers-counts[dbo.csection] = 0
    The summary said it had read x data but 0 imported
    It may be difficult for you to help me with those details but if you have any clue it will be great !
    It also may be a classical issue that i don't found any solution
    Thanks in advance
    Hi, is it possible to limit the number of copied rows?

    Hello everyone,
    I have a question about an issue am facing on. Am very new to pgloader, so any inputs will really help me.

    1) Getting an error as " the octet sequence #(195) cannot be decoded." It is because of the string getting in the source file "Women’s, Girls’, and Infants’ Cut and Sew Apparel Manufacturing ". Is there a way to handle this in load file and load data, without replacing the character in source file ?

    2) Also, the file has 1000 rows and this error happened on 425th row.. when the error happened it just stopped processing and did not continue. Is it possible to force ignore bad record & continue to next row ?

    My Load file below

    FROM INLINE with encoding 'ascii'
    Name, Value
    INTO postgresql://user:pwd@server:port/db?schema.table
    Name, Value

    WITH skip header = 1,
          batch rows = 200,
          prefetch rows = 200,
          batch size = 1024 kB,
          batch concurrency = 3,
          fields escaped by '\',
          fields terminated by '|'
    SET client_encoding to 'utf8',
        standard_conforming_strings to 'on'

    droptableifexistsschema.table; drop table if exists schema.table; ,
    dropINDEXifexistsschema.tableidx; drop INDEX if exists schema.table_idx; ,
    createtableifnotexistsschema.table(Namevarchar(80),Valuevarchar(255)); create table if not exists schema.table ( Name varchar(80), Value varchar(255) );

    029327952|"Women’s, Girls’, and Infants’ Cut and Sew Apparel Manufacturing "

    Rifhan Akram

    Hello everyone,

    I'm trying a migration from MSSQL to Postgresql. I'm getting an error as in this issue dimitri/pgloader#1018. I noticed the fix is in v3.6.2 but when I do apt-get install on Debian I get v3.6.1.

    am I missing something ? could someone please help me with this ? should I build from source? :)