Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Alex V Gonchar
    @dezconnect
    it is looks like this: https://pastebin.com/ZgfWhbzb
    Alex V Gonchar
    @dezconnect
    KABOOM!
    FATAL error: %dbsqlexec fail
    Date/time: 2019-01-24-15:49An unhandled error condition has been signalled:
    %dbsqlexec fail
    at the end before CL stacktrace
    Dimitri Fontaine
    @dimitri
    well version 3.5.2 is a little old now, so please try 3.6.1 which contains specific improvements for MS SQL support ; also review your FreeTDS configuration, which is often the problem at hand (in particular, the protocol version setup in there is important) (see http://pgloader.org for a quick intro on the topic)
    Alex V Gonchar
    @dezconnect
    Ok thanks
    Reinhardt van Rooyen
    @rvanrooy

    Hi all. I'm trying to migrate a MSSQL db and I seem to be stuck on the datetime2 columns:

    KABOOM!
    FATAL error: not supported type SYB-DATETIME2

    error

    I'm using freetds-RC1.1 and pgloader from git master 3.6.213ebde.

    can anyone point me in the right direction?

    Dimitri Fontaine
    @dimitri
    @rvanrooy please hack https://github.com/dimitri/pgloader/blob/master/src/monkey/mssql.lisp#L98 and add :syb-msdatetime2 in the list there, there's a slight chance it will just work; please then report back
    if that's all it takes, you could then open a PR that I'd be happy to accept
    Reinhardt van Rooyen
    @rvanrooy
    thanks, I'll give it a go
    Dimitri Fontaine
    @dimitri
    make sure to double check the values migrated (if any) should that edit avoid the errors you're having now, it might be that we would be handling the data in a way that was not intended, I have no idea if the syb-msdatetime2 type works like the others or not, after all
    Reinhardt van Rooyen
    @rvanrooy
    @dimitri the process definitely gets further along, it correctly scans the target rows, but gives an error status: "invalid input syntax for type timestamp with time zone: "Aug 7 2014 12:00:00:0000000" I found a github issue #431 related to msdatetime4 and and will try to add those fixes in src/sources/mssql/mssql-schema.lisp
    Ok, that seemed to work, dates looks correct at a quick glance, I'll do some more checks.
    Reinhardt van Rooyen
    @rvanrooy
    Is there a way to limit the amount of rows copied over and do incremental batches? I'm trying to copy over a huge (several hundred million rows) table, but from the debug logs, it seems that all rows are pre-fetched even though I have specified the batch rows to be fairly small. Is there a best practice to move over a large database?
    Dimitri Fontaine
    @dimitri
    see https://pgloader.readthedocs.io/en/latest/pgloader.html#with and the parameters batch rows, batch size, and prefetch rows; maybe the last one specifically here
    oh and yeah the convert()call to be used in the SELECT statement needs some attention too, in mssql-schema.lisp, good point! I guess with that you have a PR ready for improving pgloader?
    Reinhardt van Rooyen
    @rvanrooy
    correct, I'll create the PR soon
    Christoph Berg
    @df7cb
    Dimitri Fontaine
    @dimitri
    I didn't manage to reproduce that one when I tried, in a docker image, had other problems with it that I don't remember... you could experiment with building with make save and see what happens, in case that's induced by buildapp (here, I would be surprised)
    Otherwise, yeah, we need a newer sbcl would be my default attempt
    Christoph Berg
    @df7cb
    I still have next to zero lisp clue, so if no one else is going to fix it, we can't ship pgloader for stretch anymore
    Christoph Berg
    @df7cb
    Upgrading sbcl doesn't help, with sbcl 1.4.16 on stretch, I still get The symbol "SYSTEM-DEFINITION-SEARCH-FUNCTIONS" is not external in the ASDF/FIND-SYSTEM package. http://paste.debian.net/1068003/
    @dimitri: I'll stop here. Let me know what I can do to help
    Dimitri Fontaine
    @dimitri
    I'll work on reproducing that one in a local docker image
    Christoph Berg
    @df7cb
    aye
    mcgri
    @mcgri
    I have exact same issue with 'syb-datetime2' =(
    mcgri
    @mcgri
    was the issue fixed? I have 3.6.1
    John McCarthy
    @midacts
    Im running dimitri/pgloader:latest and trying to move a mssql db to a postgres db
    I've can't seem to get it to work. I'm currently stuck at this error: FATAL error: Undefined alien: "SSLeay"

    pgload file contents:

    load database
         from mssql://SA@172.17.0.2/ServerAutomation
         into postgresql://postgres:'P@@ssword'@localhost/serverautomation
    
    before load do $$ drop schema if exists dbo cascade; $$;

    pgloader command: pgloader myfile --verbose --debug

    rodolphito
    @rodolphito
    Hello
    Can pgloader convert SQL Server .sql files to PostgreSQL sql files?
    like table definitions etc
    or is it only for running databases
    rodolphito
    @rodolphito
    I'm getting errors trying to run pgloader to convert a .sql file
    2019-06-20T13:37:08.008000-07:00 LOG pgloader version "3.6.1"
    KABOOM!
    FATAL error: At
    
      -- ********************
      ^ (Line 1, Column 0, Position 0)
    
    In context KW-LOAD:
    
    While parsing KW-LOAD. Expected:
    
         the character Tab
      or the character Newline
      or the character Return
      or the character Space
      or the string "--"
      or the string "/*"
      or the string "load"
    clearly -- is one of the expected strings
    yet it explodes
    Ben Chiciudean
    @benydc
    hello
    I'm loading from csv file and importing to postgresql, how can I define default boolean value for column in target table?

    will this work:

    meta_type,state,is_imported boolean using TRUE

    this is on target columns

    Luke Snape
    @lsnape
    Hey dimitri, having trouble building the pgloader:debian docker image: Undefined foreign library: CL+SSL::LIBEAY32 when trying to run the container suggests to me it's a Debian dependency issue?
    You're using stable-slim, not a pinned version. However, I tried stretch-slim and jessie-slim and it fails for different reasons.
    Luke Snape
    @lsnape

    @dimitri not debian package versions. We think this is the culprit: cl-plus-ssl/cl-plus-ssl@f67a488

    The cl+ssl fix is not yet available via quicklisp. Any way to work around this problem in the meantime?

    Alejandro Matos
    @amatosg
    I installed pgloader from apt, when I run it it doesn't show any errors, it finishes but no data has migrated to postgres, how could I debug this?
    Varun Dutta
    @vduttadev_gitlab
    Hi anyone tried passing cast rule directly in docker run command like docker run --security-opt seccomp=unconfined --rm --name pgloader --net=host dimitri/pgloader:latest pgloader mysql://xxxx:3306/abc pgsql://xxxx:5432/abc --cast 'type bigint to int8 drop typemod' .. i tried passing all the combinations like --cast just after pgloader, in double quotes .. it is not working giving errors
    Dimitri Fontaine
    @dimitri
    Hi guys, thanks all for your interest and questions here, I'm not sure I can handle all of them, though let's try.
    About SSL, it all depends on the OS and the packaging, and Common Lisp does not make that parts as easy as I would like. I must admit I'm sometimes lots in what needs to be done for the whole thing to be happy... I know that we solved it in the Debian packaging.
    About loading data to and from SQL files, pgloader does NOT do that.
    About defining a default value for a target column, either use the CAST clause of the pgloader's command, or define your target table yourself in the target database or in the BEFORE LOAD commands.
    Max
    @Bonn93
    Hi Everyone, just wondering if anyone is still running into "SYB-MSDATETIME2" errors, I've been though the number of earlier github issues around this, running on the latest 3.6.1 and can see the usual recommendations are all applied there.
    Max
    @Bonn93
    Well, actually the errors change between versions. On the latest from Master, getting SSL errors "OpenSSL_version_num" this is the same between Github src, and the docker images.
    Brian McMahon
    @bmackattack
    First happy holidays to all! I wanted to ask if anybody else has had an issue with keeping defaults values from the original DB? I've tried casting multiple times and it doesn't get built into the DDL in the destination DB? Has anybody else had issues keeping the default value in the DDL transferred to the destination DB?