ui4j/ui4j

Web Automation for Java

aleksey

@lehins83

Hi, first of all thanks for great lib you provide. It really helps. But I have a question. I use ui4j for scrapping web pages with javascript generated content. So I need some browser. And I have url queue and processing it using ui4j. And looks like there is a memory leak. There are 3,102 instances of "com.sun.javafx.webkit.prism.WCImageDecoderImpl", and it occupies 1.5 GB of memory.

Is this me who doing something wrong?

Ozhan Duz

@ui4j

Hi @lehins83, that memory leak related with JavaFx

here is the the bug report: https://bugs.openjdk.java.net/browse/JDK-8087888

I am going to publish workaround solution to solve that memory leak

Ozhan Duz

@ui4j

I just publish workaround solution to fix the memory leak, https://github.com/ui4j/ui4j/blob/master/ui4j-sample/src/main/java/com/ui4j/sample/WorkaroundSolutionToFixMemoryLeak.java

You need to checkout latest master branch (2.1.0-SNAPSHOT) to use the sample

Let me know you if this sample enough to fix your memory leak problem

aleksey

@lehins83

Hey, @ui4j thanks a lot it works. No leaking now!

I have one more question.
If I don't use browse.show() I just use

String html = (String) page.executeScript("document.documentElement.innerHTML");

to get page's html, do I really need to use "headless" mode?

uPeekit

@uPeekit

Hey. Is there any way to use proxy in ui4j?

Ozhan Duz

@ui4j

Hi @lehins83 headless mode is optional feature for continous build etc. you could optionally use page.show() or headless mode

hi @uPeekit JavaFx WebKit browser use standard HttpUrlConnection, you could use standard Java http proxy settings: http://stackoverflow.com/questions/120797/how-do-i-set-the-proxy-to-be-used-by-the-jvm

uPeekit

@uPeekit

@ui4j so I can just set system property, that's cool. thanks :)

uPeekit

@uPeekit

what is the difference in using headless mode? seems the same for me if not call Page.show(), fx application launches anyway

Ozhan Duz

@ui4j

@uPeekit headless mode could be used for ci build, command line (without window manager)

bruce-genhot

@bruce-genhot

java.lang.ClassNotFoundException: com.sun.glass.ui.monocle.MonoclePlatformFactory

Ozhan Duz

@ui4j

Hi @bruce-genhot do not forget to add monocle dependency to your pom file: <dependency>
<groupId>org.jfxtras</groupId>
<artifactId>openjfx-monocle</artifactId>
<version>1.8.0_20</version>
</dependency>

Pat White

@patwhite

Anyone have time for a quick question about ui4j with scala, in headless mode? When I run page.captureScreen(outStream), I never get any bytes into the stream.

But, I can get the html from the response.

skaros

@skaros

hi @ui4j, could you help me with the NPE bug? i have ui4j v2.1.0 and Oracle Java 8U45, and i get NPE error on this
System.setProperty("ui4j.headless", "true");
BrowserEngine webkit = BrowserFactory.getWebKit();
Page page = webkit.navigate(url);

I just cant get it to work. Thank you

Franz Allan Valencia See

@franz-see

How do you catch any javascript error?

Franz Allan Valencia See

@franz-see

Is there a way for me to intercept all ajax requests done by ui4j?

Franz Allan Valencia See

@franz-see

Re Javascript errors : ui4j/ui4j#68

Igr Alexánder Fernández Saúco

@alexfdezsauco

Would be possible add a method to the page to load a complete document from an InputStream to be use in this way?
BrowserEngine browser = BrowserFactory.getWebKit();
Page page = browser.navigate("about:blank");
page.loadFromStream(..);

Tommaso Sotte

@sonnhy

Hi, how can i see the request/response of a page?

vvicky72

@vvicky72

Two questions pls 1) Do I have any control over image format with captureScreen? Png/tiff/jpeg? 2) Currently I have to introduce Thread.sleep() after capture or else my image is blank. Can we make the captureScreen block until the image is written out? Help appreciated in advance !!

Craig Combs

@cdiddyPerformance

Has this project been tested with web-sockets?

Guerin

@guillaumeguerin

Hello everyone :)
First of all, thanks for making such an awesome project !
Does anyone know how can I use a proxy while navigating to a url with ui-4j ?

momomo

@momomo

It doesn't seem as if navigating to http://username:password@url.com/ seems to be respected and instead we are navigating to the url without username and password. Is there a way around this?

This would if not provided normally result in a popup alert box prompting for username and password but it is outside the html context.

vojavocni

@vojavocni

Hi all, does anyone know if I can get source HTML from a webpage with ui4j?

momomo

@momomo

Is it just me or does page.hide() and page.close() not work ?

they cause future lag..

it will close the window, but next dowload using the same engine wont

kenrouzen

@kenrouzen

This message was deleted

Plab

@pabloleban

Hello guys, is there any way to simulate a click with ui4j without using JavaScript?, like a click in coordinates X and Y on the window. Cheers.

dungvv

@dungvv

@pabloleban maybe late but you can see an example in BingSearch.java, u can click on an element with java code like this:
doc.query("#sb_form_go").get().click();

Andrew Evans

@asevans48

So to get the source just use javascript. I think it is in a tutorial but page.executeScript("document.documentElement.innerHTML") should do it or in Scala page.executeScript("document.documentElement.innerHTML").asInstanceOf[String].

I have a problem. I included the correct Maven dependency and everything works fine. However, when I create a runnable jar andrun using java jdk 1.8.0_91 which I am also using from forked processes on my dev box with the exact same jdk (need multiple system proxies) the process fails. I receive Exception in thread "Thread-2" java.lang.IllegalStateException: Monocle headless platform not found
at com.ui4j.webkit.ApplicationLauncher.initMonocleHeadless(ApplicationLauncher.java:71)
at com.ui4j.webkit.ApplicationLauncher.launch(ApplicationLauncher.java:50)
at com.ui4j.webkit.WebKitBrowser$LauncherThread.run(WebKitBrowser.java:148)
Caused by: java.lang.ClassNotFoundException: com.sun.glass.ui.monocle.MonoclePlatformFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.ui4j.webkit.ApplicationLauncher.assignMonoclePlatform(ApplicationLauncher.java:87)
at com.ui4j.webkit.ApplicationLauncher.initMonocleHeadless(ApplicationLauncher.java:67)
... 2 more

Is there something else I need to do I am using -Dui4j.headless=true and have tried setting the system property.

Andrew Evans

@asevans48

I also tried -Dmonocle.platform=Headless with the bove

Andrew Evans

@asevans48

really looking to avoid xvfb-run -a

momomo

@momomo

Is it just me or do close() eventually always lead to a fatal sigserv error and a complete jvm shutdown? this makes ui4j totally unstable. it's happening all the time.

Tobias Eriksson

@tobiaseriksson

I do browser.navigate(...) and then login
after that I continue to do browser.navigate(...)
but then after a while the session seems to time out and I am logged out
how do I solve this?

♚ Sudo ツ

@badarshahzad

Hello Is there anyone to to give me few answer of my questions?

Ahmed Elkayesh

@localmaxima

I have a question please, I am using ui4j to capture screenshots of html snippets, it's always the case that the snippets fill the screen, there will be some image that will take you to another site if you click, how can I click the document without having to selected the img tag. Simply because I don't know how the snippets will be structured and it's not guaranteed there will always be <a href> tag,

I want to be able to click the middle of the viewable screen, how?

I tried page.getDocument().getElementFromPoint(x,y).get().click() where x,y are points in the middle of the viewable area, but it doesn't work

shell0dh

@shell0dh

momomo

@momomo

Is the new UI4J more stable and a real competitor to JXBrowser?

Where communities thrive

People

Repo info

Activity