Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
    Colin Alworth
    @niloc132
    right, sorry i didn't clarify that above, this is a new backend, not a frontend (the only frontends are still jdt for java or javac for java)
    i have not wandered in to the tests or what docs are present to see what kinds of use cases they imply
    Rocco De Angelis
    @rdeangelis83
    @niloc132 what steps are now necessary to get the function name property patch available in the next release? Can I take over something or help you somehow??
    Jens Nehlmeier
    @jnehlmeier
    Is there any way to send an arraybuffer (provided through JS File API, wrapped into an Uint8Array) via GWT-RPC as byte array?
    Jefferson Quesado
    @JeffQuesado_twitter

    Is there any way to send an arraybuffer (provided through JS File API, wrapped into an Uint8Array) via GWT-RPC as byte array?

    I got a workaround for this

        public static byte[] convertBytes(ArrayBuffer buffer) {
            Uint8Array array = TypedArrays.createUint8Array(buffer);
            byte[] res = new byte[array.length()];
            for (int i = 0; i < res.length; i++) {
                res[i] = (byte) (array.get(i));
            }
            return res;
        }
    don't know if it is a great approach, but it has worked
    Rocco De Angelis
    @rdeangelis83
    If I remember it correctly transferring byte arrays via GWT-RPC is not so efficient. So that we have created a customFieldSerializer
    /**
     * Tranfer of byte arrays is very memory consuming
     * --> idea: create a string out of it and transfer it in one chuck.
     * But: creating strings with many bytes does not work either properly (in Chrome and FF at least)
     * So we create several smaller strings out of the byte array and transfer that one.
     *
     */
    
    @SuppressWarnings("UnusedDeclaration")
    public class AByteArray_CustomFieldSerializer extends CustomFieldSerializer<AByteArray> {
    
        private static final int CHUNK_SIZE = 50000;
    
        @Override
        public void deserializeInstance(SerializationStreamReader serializationStreamReader, AByteArray instance) throws SerializationException {
            deserialize(serializationStreamReader, instance);
        }
    
        @Override
        public void serializeInstance(SerializationStreamWriter serializationStreamWriter, AByteArray instance) throws SerializationException {
            serialize(serializationStreamWriter, instance);
        }
    
        @SuppressWarnings("unused")
        public static void deserialize(SerializationStreamReader reader, AByteArray instance) throws SerializationException {
            // nothing here
        }
    
        public static void serialize(SerializationStreamWriter writer, AByteArray instance) throws SerializationException {
            try {
                writeBytes(writer, instance.getBytes());
            } catch (UnsupportedEncodingException e) {
                ICommonLogger.Logger.error(e.getMessage(), e);
            }
        }
    
        private static void writeBytes(SerializationStreamWriter writer, byte[] bytes) throws UnsupportedEncodingException, SerializationException {
            int length = bytes.length;
            writer.writeInt(length);
    
            int pos = 0;
            while (pos < length) {
                String string = createString(bytes, pos, Math.min(CHUNK_SIZE, length-pos));
                writer.writeString(string);
                pos += CHUNK_SIZE;
            }
        }
    
        static String createString(byte[] bytes, int from, int len) throws UnsupportedEncodingException {
            byte subBytes[] = getSubArray(bytes, from, len);
            return ABase64.createString(ABase64.encode(subBytes));
        }
    
        private static byte[] getSubArray(byte[] source, int from, int len) {
            byte[] ret = new byte[len];
            // System.arraycopy() is slow in GWT
            //noinspection ManualArrayCopy
            for (int i = 0; i < ret.length; i++) {
                ret[i] = source[from+i];
            }
            return ret;
        }
    
        @Override
        public boolean hasCustomInstantiateInstance() {
            return true;
        }
    
        public static AByteArray instantiate(SerializationStreamReader reader) throws SerializationException {
            try {
                return AByteArray.create(readByteArray(reader));
            } catch (UnsupportedEncodingException e) {
                ICommonLogger.Logger.error(e.getMessage(), e);
                return null;
            }
        }
    
        private static byte[] readByteArray(SerializationStreamReader reader) throws SerializationException, UnsupportedEncodingException {
            int size = reader.readInt();
            byte[] bytes = new byte[size];
            int pos = 0;
            while (pos < size) {
                String string = reader.readString();
                byte[] subBytes = ABase64.decode(ABase64.getBytes(string));
                append(bytes, subBytes, pos);
                pos += subBytes.length;
            }
    
            return bytes;
        }
    
        private static void append(byte[] target, byte[] source, int pos) {
            for (int i = 0; i < source.length; i++) {
                target[pos + i] = source[i];
            }
        }
    
        @Override
        public AByteArray instantiateInstance(SerializationStreamReader streamReader) throws SerializationException {
            return instantiate(streamReader);
        }
    }
    Maybe helpful ...
    Colin Alworth
    @niloc132
    @jnehlmeier i reimplemented gwt-rpc to support typedarrays and binary payloads too - the initial impl didn't bother with sending binary over xhr, only websockets, but that was just my specific requirement
    i would definitely advise sending base64 over a plain byte[] if you're worried at all about payload size (byte[] will be on average something like 3-4x to send it though it will compress well at least, whereas base64 is always about 1.3x bigger than the original data)
    @rdeangelis83 i wonder if you recall why creating strings that big didn't work? i'm guessing it was because you were implementing base64 yourself with String.fromCharCode.apply and blowing the stack? or something in gwt-rpc itself that was breaking with gigantic strings?
    Colin Alworth
    @niloc132
    @JeffQuesado_twitter welcome to gitter, did you solve your java9 module issue?
    32 replies
    Rocco De Angelis
    @rdeangelis83

    @niloc132 the code is from 2013. I don't remember exactly anymore what was the reason. Yeah we have our own base64 implementation. But If I remember correctly the problem was in ABase64.createString(byte[] bytes) which only uses new String(bytes, IStandardEncoding.STANDARD_ENCODING)

        public static String createString(byte[] bytes) throws UnsupportedEncodingException {
            return new String(bytes, IStandardEncoding.STANDARD_ENCODING);
        }

    I also remember that System.arraycopy() was always slow in GWT. But this seems to be fixed in the trunk now.

    Colin Alworth
    @niloc132
    the String ctor did have the problem i described, but i think it is fixed by building up content in batches
    Rocco De Angelis
    @rdeangelis83
    Ahhh!
    Very similar the code :)
    Colin Alworth
    @niloc132
    yeah its a familiar pattern, other js certainly has that code too... i'm sure i've written it before too
    Jens Nehlmeier
    @jnehlmeier

    Is there any way to send an arraybuffer (provided through JS File API, wrapped into an Uint8Array) via GWT-RPC as byte array?

    I got a workaround for this

    @JeffQuesado_twitter Thanks, yes that would work although I am trying to avoid copying given that the file might be large.

    Colin Alworth
    @niloc132
    it will be copied anyway, to be put on the wire, so that might not be constructive?
    this is gwt 2.8.2 code, hence mixing and matching Any[]/JsArray<Any>, and exposes an api to JS, so i have to deal with arbitrary functions that they can pass in, and memoize large amounts of ticking data from the server
    Jens Nehlmeier
    @jnehlmeier
    yeah sure, but copying just to get an array that GWT knows about seems overkill. I mean Uint8Array is already what I want, it is just that GWT-RPC does not know it can be accessed and treated like a byte[]. But I think I decided against GWT-RPC and use plain XMLHttpRequest2 + FormData. That would also give access to progress events during upload.
    Colin Alworth
    @niloc132
    and let you send binary directly
    the other fix is what @rdeangelis83 suggested, make a wrapper object that has a byte[] field, and populate that manually
    (and serialize as string base64, since you can't write binary, you have to do utf-8)
    for "zero copy" you have to pass the entire buffer right to fetch/xhr
    Rocco De Angelis
    @rdeangelis83
    @niloc132 regarding your outputFunctionNameProperty patch. Can I help you somehow to bring this changes into the next GWT release?
    Jens Nehlmeier
    @jnehlmeier
    It must be FormData since the file has some additional information attached and I don't want to make two requests. Going low level is just a bit inconvenient that's why I thought to make it work with GWT-RPC. But I totally forgot upload progress. @niloc132 does your GWT-RPC implementation emit progress events?
    Colin Alworth
    @niloc132
    @jnehlmeier no, but trivial to add - the serialization is separate from the "remote service" endpoint generation, which is still separate from the actual "how do i send", so reimplementing your fetch or xhr wrapper is a dozen lines of code, and will be generic with any given endpoint
    Jens Nehlmeier
    @jnehlmeier
    @niloc132 sounds interesting, but I guess your implementation isn't battle tested and has some rough edges given it is more or less a personal project?
    Colin Alworth
    @niloc132
    i mean its only been in production for about 2-3 years
    Jens Nehlmeier
    @jnehlmeier
    I am just a bit cautious when it comes to libs with few people behind it.
    Colin Alworth
    @niloc132
    the deephaven-core project above uses grpc, but the "enterprise" version is all my gwt-rpc via websockets
    there are bugs, but not ones that have directly affect our use case, where sending lots of dense data over short intervals is important - there is a specific generics bug that might be a headache depending on what you're doing
    Jens Nehlmeier
    @jnehlmeier
    Well just command pattern with some generics.
    Colin Alworth
    @niloc132
    it still won't be "zero" copy
    interestingly, when we moved from gwt-rpc to grpc (with flatbuffer headers and arrow buf payloads, ostensibly zero copy) for reading data, we noticed about a 5% perf improvement for ~30k row tables, but then again instead of entirely auto-generated deserialization, i had to rewrite everything by hand
    i was expecting a way bigger improvement since doing it all by hand meant less copies, more direct code (less indirection etc)
    the dominating perf issue is still "longs suck", but large rapidly ticking charts get their updates promptly, so in the end it barely mattered
    (the lesson: measure, then cut)
    dac2k9
    @dac2k9_twitter
    3 replies
    Or Goshen
    @Oberonc
    How do you format a Date() into a String these days ?
    .. in GWT ~3 that is
    Thomas Broyer
    @tbroyer
    Either use Intl.DateTimeFormat through JsInterop, or have a look at https://github.com/treblereel/gwt-datetime (available on Vertispan's repository)
    Dmitrii Tikhomirov
    @treblereel
    right now in my work i need maybe all gwt3 modules, so i forked them https://github.com/treblereel/gwtproject and published to my private repo
    Jefferson Quesado
    @JeffQuesado_twitter
    Does GWT accepts Java instanceof pattern matching? It is not stated explicitly in the changelogs, but as it was launched in may-2020 and the JDK14 had this previes back in mar-2020, maybe it has some support?
    Colin Alworth
    @niloc132
    @JeffQuesado_twitter gwt 2.9 (and current HEAD) support java 11 syntax