How to Integrate Sikuli with Selenium Webdriver
This article will help you to understand how we can integrate sikuli with selenium webdriver and how to start with automation testcases.Sikuli is another tool which is extremely useful in some cases.
Integrate Sikuli with selenium webdriver
When I started working with Selenium I used to face so many issues related to XPath and CSS so now we are going to remove all this from the script.
We can remove all these locators from the script now using another tool called Sikuli. Sikuli is another open source tool for automation
We can combine Selenium with Sikuli as well and can perform activity based on your requirement like type, click etc.
Advantage of Sikuli–
1-It is Open source tool like Selenium.
2-We can integrate with Selenium and with other tools as well.
3- An Important feature of Sikuli, it can identify object-using images/screenshots it means you can capture images for the script and can perform your operations as well depends on your requirement.
Note- in UFT this feature is known as Inside Object like Sikuli
4-Flash testing.
5-Mobile Testing.
6-Windows/Desktop application testing.
We have so many other advantages as well please go through the official docs.
Sikuli Setup for Selenium
1-Download Sikuli jars from here (My Google Driver)- Download Jars
2- Add Sikuli jar into your project.
Once Setup is complete then we can start writing selenium/sikuli scripts.
We will use some Sikuli classes that we will get once we will add that jars into the project.
1-Screen – This is a class, which will focus on the screen.
2- Pattern- This is another class, which will focus on images.
Sample Script-Integrate Sikuli with Selenium Webdriver
This script will perform following activity
1- Launch any browser and navigate to google.com (using Selenium)
2- Click on Gmail button (using Selenium)
3- On Gmail enter email, password and will click on login button (using Sikuli code)
Sikuli-Selenium Program
package demoSikuli;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.sikuli.script.FindFailed;
import org.sikuli.script.Pattern;
import org.sikuli.script.Screen;
public class demoSikul {
public static void main(String[] args) throws FindFailed, InterruptedException {
// We have to create Screen class object to access method
Screen screen = new Screen();
// Create object of Pattern class and specify the images path
Pattern image = new Pattern("C:\\gmail.PNG");
Pattern image1 = new Pattern("C:\\images\\uname.PNG");
Pattern image2 = new Pattern("C:\\images\\password.PNG");
Pattern image3 = new Pattern("C:\\images\\click.PNG");
WebDriver driver=new FirefoxDriver();
driver.manage().window().maximize();
driver.get("http://www.google.com");
screen.wait(image, 10);
// using screen object we can call click method which will accept image path and will perform //action
// This will click on gmail image on google home page
screen.click(image);
// using screen object we can call type method which will accept image path and content which //we have to type and will perform action.
// This will type on username field
screen.type(image1, "mukeshotwani@gmail.com");
//This will type of password field
screen.type(image2, "password1");
// This will click on login button
screen.click(image3);
}
}
Below images, I used in the script.
Output-
[error] ResourceLoaderBasic: checkLibsDir: libs dir is not on system path: C:\setup\libs [action] ResourceLoaderBasic: checkLibsDir: Please wait! Trying to add it to user's path [info] runcmd: reg QUERY HKCU [info] runcmd: reg QUERY HKEY_CURRENT_USER\Environment /v PATH [error] ResourceLoaderBasic: checkLibsDir: Logout and Login again! (Since libs folder is in user's path, but not activated) [error] Terminating SikuliX after a fatal error! Sorry, but it makes no sense to continue! If you do not have any idea about the error cause or solution, run again with a Debug level of 3. You might paste the output to the Q&A board.
Note- If you get error message the you have to restart your system and run the same script again
For any query join Selenium group- Join Selenium Group
Thanks for visiting my blog. Keep in touch. Have a nice day 🙂
can we run the above mentioned code in katalon
Hi Janaki,
Yes, Katalon supports all external libraries. Kindly check this Katalon with Sikuli
can anyone have idea on how to edit pdf using sikuli
Hi Janaki,
Sikuli is used for image comparison not for editing pdf.
ok Thank you for your information
You’re welcome…:)
Hi Mukesh,
Does Sikuli-script jar will work for windows, mac or is it specific to anything in particular? Also how do i integrate the sikuli-script jar file with maven.
Hi CG,
As you already mentioned it as sikuli-script jar, which means wherever you can execute jar, at those all places you can use sikuli script.
For sikuli integration with maven, you can set dependency from this maven repo page https://mvnrepository.com/artifact/org.sikuli/sikuli-api
While am running on windows 7 64 bit machine.I have received the below error message.
Exception in thread “main” FindFailed: can not find P(C://gmail.png) S: 0.7 on the screen.
Line 1574, in file Region.java
at org.sikuli.script.Region.handleFindFailed(Region.java:1574)
at org.sikuli.script.Region.wait(Region.java:1682)
at action.demoSikuli.main(demoSikuli.java:34)
Hi Priya,
It seems some issue with image path only. You can cross check.
Note- I have not tried sikuli with windows 7
Hi Mukesh,
my browser opening code is:- driver.manage().window().setSize(new Dimension(1366, 768));
Same is not working with jenkins Job, i configured the same and is working fine in eclipse and is throwing exception while running my job via Jenkins.
Below is the exception:-
INFO: Registering type java.lang.Number
[error] Location: outside any screen (33610416, 0) – subsequent actions might not work as expected
[error] Location: outside any screen (33610640, 0) – subsequent actions might not work as expected
[error] Location: outside any screen (33610640, 0) – subsequent actions might not work as expected
[error] Location: outside any screen (33610256, 0) – subsequent actions might not work as expected
[error] RobotDesktop: checkMousePosition: should be L(33610640,0)
but after move is L(33610256,0)
Possible cause in case you did not touch the mouse while script was running:
Mouse actions are blocked generally or by the frontmost application.
You might try to run the SikuliX stuff as admin.
not sure on this.
which tool we can use to taking screenshots?
snipping tool for windows
Hi Mukesh,
Can you please share some code which perform click on windows pop-up which appears during file download.
use click () method.
in lib folder showing like empty file and i attched skuli scrpt jar in java project but i am getting same error
Hi Yogesh,
Please find new doc and follow doc carefully.
http://sikulix.com/
http://sikulix-2014.readthedocs.io/en/latest/index.html
Hi Mukesh, thank you so much for this very useful and informative tutorial.
But I do have a question – Is there a way to compare images/screenshots using Sikuli and Webdriver?( to verify if that image is present in webpage).
Yes Chitra, we can do that. I dont have tutorial in this but you can check sikuli doc then you will get doc and details as well.
Hi Mukesh,
Following is the Sikuli code I used to read from Excel in Hybrid Framework:
Screen screen = new Screen() Pattern Img1 = new Pattern(sTI1);//sTestInput – read from Excel
//To high light the Image
Match m = screen.find(Img1);
m.highlight(5);
//To Click the image
screen.click(screen.wait(Img1, 10)); Thread.sleep(1000);
I saved my screenshot in C:\Contact_us.PNG.
So my test input in excel is: C://Contact_us.PNG
Hope this helps someone who is new to Sikuli and want to explore its possibilities with Hybrid Framework like me.
Many thanks to you for introducing this tool and be an inspiration for testers like me.
Edit: Screen screen = new Screen();
Missed semi colon on the end:)
Thanks Chitra this will really help others. 🙂
hi,
i have learned selenium webdriver through your website.thanx for that. Learning silkuli is howmuch important.this blog is enough for learning silkuli. i wanr your suggestion.if not plz mention a link
Hey Sudhir,
Sikuli is also big tool but for Selenium point of view we wont use much so you can still explore from your side but from Selenium point of view you should know how to handle windows activity using Sikuli
This is awesome. thanks a lot mukesh.
Hey Saurabh your most welcome. Keep visiting and let me know if any help from my side.
Hi Mukesh,
Thanks for the great info and its very useful.
I tried using the same above but am getting displayed with below error.
P(C:\Users\vskkr1\Desktop\Sikuli\url.PNG) S: 0.7
Exception in thread “main” FindFailed: can not find P(C:\Users\vskkr1\Desktop\Sikuli\url.PNG) S: 0.7 on the screen.
Line 1574, in file Region.java
at org.sikuli.script.Region.handleFindFailed(Region.java:1574)
at org.sikuli.script.Region.wait(Region.java:1682)
at org.sikuli.script.Region.find(Region.java:1590)
at org.sikuli.script.Region.getLocationFromTarget(Region.java:1991)
at org.sikuli.script.Region.click(Region.java:2197)
at org.sikuli.script.Region.keyin(Region.java:2718)
at org.sikuli.script.Region.type(Region.java:2678)
at autoit.Calculator.main(Calculator.java:48)
Could u pls help me if u have any idea on the same
Hi Kishore,
I can see th images which you specified is not valid. Kindly check the path again.
Hope you have downloaded successfully. Kindly use http://www.sikulix.com/quickstart.html#qs2 for more details.