Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • 20:40
    iains commented #684
  • 20:38
    feeley reopened #684
  • 20:38
    feeley commented #684
  • 19:21
    iains closed #684
  • 19:21
    iains commented #684
  • 18:43
    iains opened #684
  • 18:41
    iains closed #683
  • 18:41
    iains commented #683
  • 18:40
    iains synchronize #683
  • 18:14
    feeley commented #683
  • 17:04
    iains commented #683
  • 16:17
    iains opened #683
  • 14:17
    feeley commented #681
  • 12:59
    gambiteer commented #681
  • 02:15
    gambiteer commented on a790243
  • Apr 21 20:46
    feeley commented #681
  • Apr 21 19:56
    gambiteer commented #681
  • Apr 21 19:52
    Windows-mingw build of CI run 771806576 failed
  • Apr 21 19:25

    feeley on master

    Fix GVM jump instruction mishan… Universal backend: avoid possib… (compare)

  • Apr 21 15:15
    gambiteer commented #681
Marc Feeley
@feeley
let me add that it might be useful to store the library definition in the compiled modules so that a program that is statically linked can do dynamic imports (as in (eval ‘(import …))) of the linked modules without having to go to the filesystem
Jürgen Geßwein
@jgesswein
Sounds good. I need some more advice on how to do that. Currently I am not building a statically linked application as compiling takes also quite some time… ;-)
Marc Feeley
@feeley

the multiple imports of _test are due to the way the macros are written… for example test-equal is defined like this :

(##define-syntax test-equal       (lambda (src)
                                    (##import _test/test-expand)
                                    (test-expand src 'test-equal)))

so every call to test-equal will cause a import to be processed

clearly a cache would help, but maybe a rewrite of the macros would be sufficient
I suspect you have a program with a large number of tests...
Jürgen Geßwein
@jgesswein
Is 676 a large number? I am currently not using library _test but some home-grown stuff for historial reasons.
Marc Feeley
@feeley
sorry my analysis is wrong… the import is just done once per macro definition (not every macro call)
can I see your test library?
Jürgen Geßwein
@jgesswein
I’ll take a look at the 36MB file and give you some number on how often the libraries are loaded. You can then decide with more confidence on a cache.
Sure, why not. I drop you an e-mail. Is your university address OK?
Marc Feeley
@feeley
yes
36MB/676 is about 50KB per test… (if each test is generating some imports)… 50KB would be a lot!
Jürgen Geßwein
@jgesswein
Well that file is just for loading the libraries. The tests did not yet run. I create function with tests and run those functions. This allows for a progress bar.
Marc Feeley
@feeley
I see… but still each macro call may be causing some imports… and these are processed at macro expansion time (before execution)
Jürgen Geßwein
@jgesswein
I think I do not do this. But you may have a look at the module.
amirouche
@amirouche

Here is an extract from an article I am writing:

I experimented with Gambit JavaScript target, looked at the
benchmarks (but I did
not run them myself, yet), read on the Termite library that is inspired from
Erlang, and the upcoming Scheme Infix eXpression (SIX) already
available in master branch: In the years to come, Gambit Scheme
will become the goto programming stack for industrial use
. Here is
why:

  • Gambit Scheme is a Scheme;

  • Termite is what you need to build an application distributed in a
    controlled environnement, without requiring a change of programming
    language (see Erlang);

  • Gambit can also target the web browser;

Otherwise stated: we will be able to build full-stack applications
that can scale painlessly thanks to Termite, with a single great
programming language, the same compiler, and interpreter backend-side
and frontend-side.

Anyway, even if I am bit scared, I will try to use Gambit.
Jürgen Geßwein
@jgesswein
Something that also makes using the module system hard is that Gambit does not complain if a library contains identifiers that it cannot resolve. At least a warning would be helpful. Especially if an identifier in export is not defined.
amirouche
@amirouche
that is a known (by me) problem, and fwiw not the only one, for instance the module system requires to explicitly export dependent forms of a macro. And they are some things that may be trivial, but very great such as a directory contains all the files that defines the library unlike the foo.sld that includes foo/body.scm. Also, the ability to specify a git repository is great, it alleviates the need for a package management tool.
AFAIU it is possible to layer on top of Gambit the project called unsyntax (or fork) it support all macro systems (I will give it try at some point).
Speaking of library system and JS target, it would be easier for me if it produced a single .js file.
Marc Feeley
@feeley
@amirouche nice to hear your praise of Gambit (current and future)!
what do you mean “if it produced a single .js file”? that is what is produced by -exe… and if you want that wrapped in a trivial HTML you can -exe -o foo.html
amirouche
@amirouche
Last time I tried to compile a program.scm with (import (foobar)), there was several messages in the browser console about failed attempts to retrieve foobar.sld over the network.
I use -:r7rs
Marc Feeley
@feeley
if you do a static link then the dynamic module loading will not happen (unless you forgot to link the module in!)
amirouche
@amirouche
I will look into static link.
Marc Feeley
@feeley
one issue with the current static linker is that you have to help it out with locating the modules (ideally it should be automatic and this is on my TODO)… also you have to use the -nopreload linker option
amirouche
@amirouche
Re JS promise, does the resolution of promise block or pause only the current thread? Hence it would be possible to spawn two threads and do a fetch in each of them?
Related question is: is call/cc fast? In my prototype, I use call/cc to implement "non-colored async that look sync" similar to what Gambit has builtin with the lightweight threads and network io.
Ultimately, what would be the best way to implement network io browser side without blocking.
Marc Feeley
@feeley
yes waiting for promise results is done on a thread by thread basis (that’s one of the difficulties in implementing this and frankly it would lose a lot of value if it wasn’t that way)
yes call/cc is “fast” (the Scheme thread scheduler uses call/cc or rather continuation-capture to switch threads)
what is “network i/o”? fetch?
you can spawn N Scheme threads, each doing a JS fetch and it will all be concurrent (it is one of the examples in the paper I shared with you, which will be presented at ELS21 by the way)
amirouche
@amirouche
By "network i/o" I mean fetch and websockets.
Marc Feeley
@feeley
all of that currently works well

try this on https://gambitscheme.org/try

(define-syntax future
  (lambda (stx)
    (syntax-case stx ()
      ((future expr)
       #'(thread (lambda () expr))))))

(define touch thread-join!)

(define (pmap f lst)   ;; "parallel" map
  (map touch (map (lambda (x) (future (f x))) lst)))

(define memo
  (string-append
   "Scheme_-_An_interpreter_for_extended_"
   "lambda_calculus.djvu"))

(define (page n)
  (string-append
   "https://upload.wikimedia.org/wikipedia"
   "/commons/thumb/1/1e/" memo
   "/page" (number->string n) "-593px-" memo ".jpg"))

(define (fetch-blob url)
  \fetch(`url).then(function (r) { return r.blob(); }))

(define (->URL blob)
  \URL.createObjectURL(`blob))

(define (show url)
  \document.body.insertAdjacentHTML(
   "beforeend",
   "<img src='"+(`url)+"' width=200px>"))

(define images
  (pmap (lambda (n) (->URL (fetch-blob (page n))))
        (iota 43 1)))

(for-each show images)

you should see all the pages appear quickly… if you replace pmap by map it will be 10x slower

amirouche
@amirouche
Indeed! Thanks again for your precious time.
Drew Crampsie
@drewc
OMG!! I have not tried that part yet but am about to ... we auto-resolve promises? So the async appears synchronous in (use-result-value (promise-value))??
drewc @drewc REPL'ies to himself ...
Drew Crampsie
@drewc
Oh that is very very cool.
amirouche
@amirouche
It is not useful for my immediate need of Gambit, but it will inspire more confidence going forward: Is there a readily available documentation regarding the Universal backend? Otherwise, what is the core code or entry point into the Universal backend?
Regarding the infix notion called SIX, it is already usable for interfacing with C? Otherwise, is it planned or an interesting feature ?
amirouche
@amirouche
Instead of "inspire confidence" it should be written "increase confidence".
Marc Feeley
@feeley

@amirouche it depends what kind of documentation you are looking for… there are academic papers about how the universal backend works (see for example https://www.researchgate.net/publication/304551221_Compiling_for_multi-language_task_migration). There is no “software developper” documentation however. Here’s a quick overview. The files implementing the universal backend are gsc/_t-univ-*.scm . In essence they implement a translator from the GVM (Gambit Virtual Machine) code and the target language (JavaScript, Python, etc). There are 2 main entry point to the universal backend: the procedure univ-dump-code (which implements GVM -> target) and the procedure univ-link (which implements the static module linker). Both of these procedures, and other methods of the backend are contained in a target object that is initialized by the univ-setup procedure:

(univ-setup 'js     '((".js"   . JavaScript))  '()        '())
(univ-setup 'python '((".py"   . Python))      '((pre3))  '())
(univ-setup 'ruby   '((".rb"   . Ruby))        '()        '())
(univ-setup 'php    '((".php"  . PHP))         '((pre53)) '())

These calls define the targets that can be used on the gsc command line, i.e. -target js, -target python, etc The fields of the target object are documented in gsc/_back.scm. For example:

;; nb-regs      Integer denoting the maximum number of GVM registers
;;              that should be used when generating GVM code for this
;;              target machine.
;;
;; nb-arg-regs  Integer denoting the maximum number of procedure call
;;              arguments that are passed in GVM registers.
;;
;; compactness  Integer denoting the level of compactness of the
;;              generated code.  Levels from 0 to 5 cause the
;;              generation of increasingly compact code with little or
;;              no impact on execution speed.  Lower values tend to
;;              make the generated code more humanly readable.  Above
;;              a level of 5 the compiler will trade execution speed
;;              for saving code space.  The detailed meaning of this
;;              option depends on the target and some targets may
;;              ignore it.

The universal backend implements the code generation for multiple target languages by using code generation macros. These have names starting with a ^, for example (^if test true [false]) generates a 1 or 2 branch “if” construct in the target language and (^ X Y Z) concatenates the target code X, Y and Z. The ^if macro is implemented through the univ-emit-if procedure which dispatches on the target language:

(define (univ-emit-if ctx test true #!optional (false #f))
  (case (target-name (ctx-target ctx))

    ((js php java)
     (^ "if (" test ") {\n"
        (^indent true)
        (if false
            (^ "} else {\n"
               (^indent false))
            (^))
        "}\n"))

    ((python)
     (^ "if " test ":\n"
        (^indent true)
        (if false
            (^ "else:\n"
                  (^indent false))
            (^))))

    ((ruby)
     (^ "if " test "\n"
        (^indent true)
        (if false
            (^ "else\n"
               (^indent false))
            (^))
        "end\n"))

    ((go)
     (^ "if " test " {\n"
        (^indent true)
        (if false
            (^ "} else {\n"
               (^indent false))
            (^))
        "}\n"))

    (else
     (compiler-internal-error
      "univ-emit-if, unknown target"))))

The implementation of Scheme primitives is done through the univ-define-prim macro. For example here is the implementation of ##fxmin that returns the minimum of two fixnums:

(univ-define-prim "##fxmin" #t
  (make-translated-operand-generator
   (lambda (ctx return arg1 arg2)
     (return (^if-expr 'scmobj
                       (^< (^fixnum-unbox arg1) (^fixnum-unbox arg2))
                       arg1
                       arg2)))))
Marc Feeley
@feeley
The ^if-expr macro generates conditional expressions. Note that the first parameter is a type (the type of the value returned by the conditional expression) because some targets need to have this information, namely the go target implements conditional expressions using a function call:
(define (univ-emit-if-expr ctx type expr1 expr2 expr3)
  (case (target-name (ctx-target ctx))

    ((js ruby java)
     (^ expr1 " ? " expr2 " : " expr3))

    ((php)
     (^parens (^ expr1 " ? " expr2 " : " expr3)))

    ((python)
     (^ expr2 " if " expr1 " else " expr3))

    ((go)
     (^apply (univ-emit-fn-decl
              ctx
              #f
              type
              '()
              (^if expr1
                   (^return expr2)
                   (^return expr3)))
             '()))

    (else
     (compiler-internal-error
      "univ-emit-if-expr, unknown target"))))
amirouche
@amirouche
I noted those information and others in the following document: https://github.com/amirouche/gambit-scheme-pawns#readme
I would be happy if someone could review at least the last three questions: https://github.com/amirouche/gambit-scheme-pawns#how-does-termite-compare-to-concurrentml