Using SikuliX to handle file upload dialog window

In this article we are going to use SikuliX to complete file upload operation. Let's make an assumption that we are working with Selenium WebDriver and we need to automate file uploading. Selenium will do the job to open a browser, navigate to https://uploadfiles.io, open file upload window and Sikuli will select file to upload.

Prerequisites are:
- jdk;
- maven;
We are going to use Ubuntu tat is why we need to install two more packages:
- libopencv;
- tesseract.

On Ubuntu these packages can be installed via package manager apt-get:
sudo apt-get install libopencv-dev
sudo apt-get install tesseract-ocr

Then we need to add Selenium and SikuliX dependencies to project:

<dependency>
    <groupId>org.seleniumhq.selenium</groupId>
    <artifactId>selenium-java</artifactId>
    <version>2.53.1</version>
</dependency>
<dependency>
    <groupId>com.sikulix</groupId>
    <artifactId>sikulixapi</artifactId>
    <version>1.1.0</version>
</dependency>

No we need to create screenshots for Sikuli. File upload window is:

With the help of Sikuli we will prepare a set of actions to wait for Documents icon to appear, then click it, select file to upload and click 'Open' button. For this we need to make three screenshots:

Let's store our screenshots for current project, for example under 'resources' directory.

And now the most interesting part -- script:

 WebDriver driver = new FirefoxDriver();
 driver.manage().timeouts().implicitlyWait(30, TimeUnit.SECONDS);
 driver.manage().window().maximize();

 driver.get("https://uploadfiles.io/");
 driver.findElement(By.id("dropzone")).click();
 Screen s = new Screen();
 try {
    s.wait(pathToImgResources + "/documents-icon.png");
    s.click(pathToImgResources + "/documents-icon.png");
    s.click(pathToImgResources + "/test-document.png");
    s.click(pathToImgResources + "/open-button.png");
   } catch (FindFailed e) {
     e.printStackTrace();
   }

 Assert.assertEquals(
           driver.findElement(By.xpath("//div[@id='edit']/h3")).getText(),
           "Done! Your file is available at:");

That's it! Sikuli might be really handy when you need to operate on a control which can't be accessed with Selenium API. And we just saw an example of such a case.