by

Where communities thrive


  • Join over 1.5M+ people
  • Join over 100K+ communities
  • Free without limits
  • Create your own community
People
Repo info
Activity
  • Jul 06 09:23
    seseso commented #376
  • Jul 06 08:56
    seseso synchronize #376
  • Jul 05 08:17
    benas unlabeled #373
  • Jul 05 08:17
    benas closed #373
  • Jul 05 08:17
    benas commented #373
  • Jul 05 06:31
    vinay-sd17 commented #373
  • Jul 04 16:55
    benas labeled #376
  • Jul 04 16:55
    benas commented #376
  • Jul 04 16:12
    benas labeled #373
  • Jul 04 16:12
    benas commented #373
  • Jul 04 16:09
    benas edited #373
  • Jul 04 16:09
    benas labeled #375
  • Jul 04 16:09
    benas closed #375
  • Jul 04 16:09
    benas commented #375
  • Jul 04 16:06
    benas edited #375
  • Jul 04 16:06
    benas labeled #377
  • Jul 04 16:05
    benas commented #377
  • Jul 04 15:51

    dependabot[bot] on maven

    (compare)

  • Jul 04 15:51

    benas on master

    Bump quartz from 2.3.0 to 2.3.2… (compare)

  • Jul 04 15:51
    benas closed #378
Shoaib R Khan
@shoaibjdev
Hi, is there a UI for Easy Job like Spring batch admin?
Mahmoud Ben Hassine
@benas
Hi, yes, take a look at https://github.com/j-easy/easy-jobs
it's not specific for easy batch, but can administer easy batch jobs
Shoaib R Khan
@shoaibjdev
Thank you
arun0287
@arun0287
Hi.. just thinking on what we need to do to make this distributed, fault tolerant, monitoring and reporting enabled. Any leads or ideas??
arun0287
@arun0287
@benas kindly suggest/help
Mahmoud Ben Hassine
@benas
what we need to do to make this distributed, fault tolerant, monitoring and reporting enabled
In this case, I would recommand using Spring Batch :smile:
张成
@Sailfishc
@benas Hello, i will use easy-batch, but my project use mybatis, do you have some way make me quick start
Mahmoud Ben Hassine
@benas
@Sailfishc There is no built-in support for mybatis in easy batch. You would need to write a custom reader/writer for that.
张成
@Sailfishc
@benas thank you
ramonster
@ramonster
Hi all. I have a task of using easy batch to do some processing. But i need to do some work before performing the rest of the batch Job. I basically have to download some files before I can do the processing bit. But i can't seem to figure out how to do this as part of the Job
thanks for your help guys
Mahmoud Ben Hassine
@benas
@ramonster I guess JobListener is what you are looking for.
ramonster
@ramonster
Cool, thanks Mahmoud, i'll have a look into it
Appreciate the quick response
AskBhushan
@Bhushan01561872_twitter

I have odd need, I have multiple input csv files, i need to read each one of them and based on the file name, enrich & add few more columns to each record/line of csv and finally persist that csv into db, How do get the file name in Writer or Mapper?

Basically, based on the file name, some computation needed to create few more columns in the input csv, e.g. if file name is FILENAME_datetime_userid_productcode.csv, then date time and product code needs to be used for further calculation and the userid needs to be used for linking few other incoming files.

I am planning to use 1> MultiFlatFileRecordReader and read incoming files, 2> JdbcRecordMapper for converting record into pojo and then user JDBCRecordWriter, But not sure how to read the incoming file name in mapper. can you please suggest?

Mahmoud Ben Hassine
@benas
The JdbcRecordMapper is used when you read data from a database, but in your case you are reading data from flat files. The easiest way I can think of is to create an object that holds the file name and which you share between the mapper and writer. Hope this idea helps!
romeoro
@romeoro
Hi together, i really loved easy batch on first sight compared to the more complex spring batch. I want to use it in microservice cloud environment where a batch job would need to be "synced" between multiple instances of the same service running. Spring batch can handle this via locks on the shared database. Is there something similar in easy batch? Couldn't find something :(
To be more precise. A single database entry should not be processed twice, when it is read and deleted in the same job.
Mahmoud Ben Hassine
@benas
There is no central JobRepository concept in Easy Batch. Try to make sure that each job works on a different set of data and you would be safe to go.
romeoro
@romeoro
Hm not sure if that is an option. Thanks anyway!
hbell-consulting
@hbell-consulting
Hi @benas can you please give the name of the tool that help you to draw this : https://raw.githubusercontent.com/wiki/j-easy/easy-batch/images/batch-processing.png Thanks a lot
Mahmoud Ben Hassine
@benas
Hi @hbell-consulting I use http://diagrammix.com to draw these diagrams. Nice heinh? :smile:
hbell-consulting
@hbell-consulting
@benas Thanks, the draws ara awesome !!
Mahmoud Ben Hassine
@benas
Thank you!
hbell-consulting
@hbell-consulting
I'm currently working on projects with Spring Batch and tonight by chance, I discover easy-batch, it's really top what you've done ^^
Mahmoud Ben Hassine
@benas
Oh great thank you! glad to hear that. I work on both sides so feel free to ask if you need help.
hbell-consulting
@hbell-consulting
Thank you vey much !!
Mahmoud Ben Hassine
@benas

Hi all,

Easy Batch v6.0 has been released a few days ago: https://github.com/j-easy/easy-batch/releases/tag/easy-batch-6.0.0. This is probably the best release ever since the beginning. It is based on Java 8 and provides some nice new features like batch scanning (j-easy/easy-batch#363). I would like to thank all contributors who helped making this release possible!

Easy Batch is now trusted by some serious companies like Splunk (see https://docs.splunk.com/Documentation/DBX/3.2.0/ReleaseNotes/easybatch) and others (https://speakerdeck.com/benas/easy-batch?slide=41). This is very rewarding and I'm proud to see this happening. Easy Batch needs more users so if you have used it successfully, do not hesitate to tweet about it, star it on github, write a blog post about it and spread the word!

Best regards

czfh
@czfh_gitlab
@benas and team, could you give me some advice how to integrate with Message Queue(For example, Rabbit MQ, Kafka, etc) from Easy Batch.
And how to config or setup Easy Batch v4.0 to make it work in Transaction mode when integrated with MQ. Thanks in advance.
Mahmoud Ben Hassine
@benas
@czfh_gitlab I guess you opened this issue: j-easy/easy-batch#370. I added an answer there, hope it helps.
repolevedavaj
@repolevedavaj
Hi, is it intentionally, that the RecordWriter/Batch classes cannot be used with generics? Due this, i have to cast to my record type in the writer. Or do I miss something here? (I started working with this library yesterday ;) )
Mahmoud Ben Hassine
@benas
Record<P> is generic: https://github.com/j-easy/easy-batch/blob/master/easy-batch-core/src/main/java/org/jeasy/batch/core/record/Record.java#L56. As far as the record writer is concerned, it has to write records (regardless of their payload), so it does not have to be generic. Why do you need a cast? If you share an example, I can help.
czfh
@czfh_gitlab
@benas Yes, really appreciate your practical guide . And could you pls give me some advice how to create cache connection factory or add a beforeProcessing method after one batch writer. Thanks in advance.
repolevedavaj
@repolevedavaj
@benas may be i understand something wrong but if i receive a Batch in my writer, i do not know the type of the payload. If i have a writer which writes XML documents into files and the XML Document is passed as payload, i have to cast the payload in the writer it into my concrete type. If the Batch class would be generic, i could declare it already with the correct type and do not have to cast.
Mahmoud Ben Hassine
@benas
@repolevedavaj RecordWriter does not have to be generic. Batch however could be made generic with the same type the payload of its records. Please open an issue and I will take a look. Thank you upfront.
@czfh_gitlab A job listener is a good candidate where you can populate/clear a cache (See https://github.com/j-easy/easy-batch/wiki/listeners#job-listener). Then you can use the cache where appropriate in your job (processor, writer, etc).
czfh
@czfh_gitlab
@benas Understood. Thanks. Want to create one JMS connection factory to cross multi services based on Spring framework. For example, create one JMS connection, session, etc by beforeWriter. And then destroy them in afterWriter. I wonder if it is feasible when working with Easy Batch 4.x. version. :-)
Mahmoud Ben Hassine
@benas
@czfh_gitlab The closest you can get with v4 is the PipelineListener. But I really recommend you to upgrade to the latest version.
czfh
@czfh_gitlab
Great! Got it, but this is one legacy project, and I cannot decide to do that by myself. :-( . I am looking for some possible solution examples as XP.
czfh
@czfh_gitlab
It's solved. Thank you so much @benas
Abhishek Narayan
@anarayn
Is there a way to read and write to mongodb
I meant builtin reader and writer for MongoDB
Mahmoud Ben Hassine
@benas
@anarayn There used to be support for mongodb in v5 but it has been removed in the latest v6. You can find more details about the reason for that in j-easy/easy-batch#354.
Abhishek Narayan
@anarayn
@benas Thanks! it's helpful.
Ryan Gross
@Ryan-Gross1993

Hello all,

I recently found easy-batch as I'm trying to read and process records from an .xlsx file.
I'm using it for a personal project. :)

Mahmoud Ben Hassine
@benas
Hi @Ryan-Gross1993 , good to know :smile: You might have already seen/used the msexcel module, but in case you didn't, here it is: https://github.com/j-easy/easy-batch/tree/master/easy-batch-extensions/easy-batch-msexcel.