Questions tagged with AWS Device Farm

Is there a way to rename session ID for AWS Device Farm desktop browser testing ? It's hard to debug tests with random session ID. It's more reasonable to provide option to replace test ID or name in leu of session ID

Command used in GitHub Actions to download APK from expo: latest_build=$(npx eas-cli build:list --status="finished" --distribution="store" --json | jq '[.[] | select(.releaseChannel=="development")][1].artifacts.buildUrl') Command used in GitHub Actions for create-upload and upload in device farm response_upload_app=$(aws devicefarm create-upload --project-arn $DEV_PROJECT_ARN --name latest_build.apk --type ANDROID_APP) curl -T latest_build.apk $url_upload_app Same command when run locally in terminal when the APK is available in a folder, works perfectly fine. Also, at times, when running in local terminal, it was giving request timeout error This is the error log in GitHub Actions in when running get-upload command for the corresponding create upload arn in device farm: "metadata": "{\"errorMessageUrl\":\"https://docs.aws.amazon.com/console/devicefarm/ANDROID_APP_AAPT_DEBUG_BADGING_FAILED\",\"errorMessage\":\"We could not extract information about your Android application package. Please verify that the application package is valid by running the command \\\"aapt debug badging <path to your test package>\\\", and try again after the command does not print any error.\",\"errorCode\":\"ANDROID_APP_AAPT_DEBUG_BADGING_FAILED\"} Debugging done so far: Ran this (aapt debug badging <path to apk>/latest_build.apk ) and was able to get package information correctly

I'm trying to get a simple test case working with XCTest UI on AWS Device Farm. I followed the steps to create the two .ipa files and uploaded them into a device farm project without any issues. I tried using both standard and custom environments and used the defaults for the remaining run options with similar results: the setup and tear down suites pass but the tests suite fails. The test suite is as simple as possible as they are the stubs created when an XCTest UI target is added to the Xcode project. The test ("testExample") runs fine in Xcode as do the other stubbed tests (testLaunch and testLaunchPerformance). I'm assuming that Device Farm will pick up all the available tests (those test functions starting with "test") in a similar fashion that Xcode identifies testable functions so I don't do anything special to identify those to Device Farm. As I stated, I don't embellish the stubbed out tests so testExample() simply launches the app and that's it. Still, I would expect the test to pass as it does in Xcode. Are there some basic gotchas in configuring test runs or Xcode projects that would prevent a test function from succeeding?

Hi, I'm looking for the IP ranges of the Desktop Browser service from Device Farm. I have currently added the following ranges to my VPN: ``` 100.20.244.103/32 44.225.225.32/32 44.227.49.133/32 ``` But some sessions have an IP outside these ones. Could I get an updated IP ranges for the Desktop Browser service? Thank you and have a nice day, Davide

Hi, I tried to run ./gradlew deviceFarmUpload but it failed because I am not authorized to perform: devicefarm:ListProjects with an explicit deny in an identity-based policy. I checked with my admin and he stated that I am in a user group which has the AWSDeviceFarmFullAccess policy attached to the group. arn:aws:iam::aws:policy/AWSDeviceFarmFullAccess { "Version": "2012-10-17", "Statement": [ { "Action": [ "devicefarm:*" ], "Effect": "Allow", "Resource": "*" } ] } Can I get help with this issue? Thank you so much.

When I use the iOSDevice Farm Python Appium test case officially provided by AWS, my iOS test has no performance test report, but when I use the Android test case, I can get the performance test report? What is the reason for this? What should I do? manage?

We are trying test our application in AWS Devicefarm. We have created Appium Java Testng tests, which would run on devicefarm real devices. We want to verify the battery usage when we use the app. We are planning to achieve this by comparing the battery levels at start and end of test. We are able to extract the device battery level using Appium, but in most cases the device appears to be in charging state. Is there a way to set the battery level in device farm or pick up devices which are not in charging state.

The requirement is to run a few voice utterances (Eg: OK google or Alexa, Hows the whether) in device farm and see if I can get a response on the phone using Appium. If I have the voice utterances maintained in mp3 format, can the output be played from the machine? so the phone can respond to the request?

Hi all. i got this error while using aws device farm while try to run java appuim testng project: Unable to create a new remote session. Please check the server log for more details. Original error: An unknown server-side error occurred while processing the command. Original error: Cannot start the 'com.XXX.staging' application. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting. Original error: 'com.XXX.YYY.ui.activities.MainActivity' or 'com.XXX.staging.com.YYY.core.ui.activities.MainActivity' never started. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03' System info: host: 'ip-111-11-11-11', ip: '111.11.11.61', os.name: 'Linux', os.arch: 'amd64', os.version: '3.13.0-139-generic', java.version: '1.8.0_151' Driver info: driver.version: AndroidDriver locally it's run well. i sent this caps for aws server: ------------- AppiumDriver driver; final String URL_STRING = "http://127.0.0.1:4723/wd/hub"; URL url1 = new URL(URL_STRING); --------------- DesiredCapabilities desiredCapabilities = new DesiredCapabilities(); driver = new AndroidDriver(url1, desiredCapabilities); please help, thanks head! Oron.

We are using AWS device farm to execute Android and IOS tests, after we pass the 100 test cases border we start to see an error on Android, Appium is not able to find a free port, `Original error: Cannot find any free port in range 8200..8299}`. I'm sure that in our code we are calling `driver.quit` in order to release the ports, also before each test I run an adb command to check current status and there is no ADB ports used, so Im worry if maybe the problem is with the ADB version used in device farm, we are using custom environments and Appium 1.22.3 with Java.

I found this on DeviceFarm documentation: https://docs.aws.amazon.com/devicefarm/latest/developerguide/limits.html *There is no limit to the number of devices that you can include in a test run. However, the maximum number of devices that Device Farm will test simultaneously during a test run is five. (This number can be increased upon request.)* How can I increase the number of devices? It doesn't specify where or how to request it... I basically need this to run more test in parallel using the metered payment plan. Right now even if we have multiple tests making requests on the test project in device farm, only 5 tests are executed at a time, the remaining are pending.

Hi everyone, I have the same question with the author from https://stackoverflow.com/questions/45365413/how-to-connect-via-adb-to-a-remote-session-android-device-on-aws-device-farm. It seems like the answer is archived so please let me put it here again. """ Can I connect to Android device during a remote session via adb? I would need to connect to the Android device via adb( using tcpip mode) but I presume, I can't connect to a specific device via a public IP(?) so how can I connect from a ubuntu-server ec2 instance or a local machine to a device farm Android device during a remote session? During the remote access session, the device has private ip, offcourse (e.g. 192.168.1.10). Would using a VPC help in this regard? Any help/pointers would be greatly appreciated. Thanks. """" Hope for your help! Thank you!

When testing my app on an AWS Device Farm physical device via automation, is it possible to upgrade the version of chrome it comes with by default? For example when using an Android 8 device, the default version of chrome is a few upgrades behind. It would be very useful to be able to upgrade chrome therefore to the version that current Android 8 users will be using. If not possible on a "public" device, is this possible with a "private" one?

Hi, I am running my test on device farm. From the test specification log, it shows that my test has passed. However from the video recording, it is always stuck on the default mobile homescreen. How do i get the video recording to capture the test interaction instead of just the homescreen. thanks! ``` [TestNG] INVOKING CONFIGURATION: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - @BeforeTest Tests.LoginTest.setUpPage() [TestNG] PASSED CONFIGURATION: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - @BeforeTest Tests.LoginTest.setUpPage() finished in 118 ms [Invoker 1508395126] Keeping method TestBase.navigateTo()[pri:0, instance:Tests.LoginTest@15975490] for class [TestClass name=class Tests.LoginTest] [Invoker 1508395126] Invoking @BeforeClass TestBase.navigateTo()[pri:0, instance:Tests.LoginTest@15975490] [TestNG] INVOKING CONFIGURATION: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - @BeforeClass Tests.AbstractBaseTests.TestBase.navigateTo() [TestNG] PASSED CONFIGURATION: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - @BeforeClass Tests.AbstractBaseTests.TestBase.navigateTo() finished in 2846 ms [Invoker 1508395126] Keeping method LoginTest.logOut()[pri:0, instance:Tests.LoginTest@15975490] for class [TestClass name=class Tests.LoginTest] [TestNG] INVOKING: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - Tests.LoginTest.loginFail() [Invoker 1508395126] Invoking Tests.LoginTest.loginFail [TestNG] PASSED: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - Tests.LoginTest.loginFail() finished in 7446 ms [Invoker 1508395126] Keeping method LoginTest.logOut()[pri:0, instance:Tests.LoginTest@15975490] for class [TestClass name=class Tests.LoginTest] [Invoker 1508395126] Invoking @AfterMethod LoginTest.logOut()[pri:0, instance:Tests.LoginTest@15975490] [TestNG] INVOKING CONFIGURATION: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - @AfterMethod Tests.LoginTest.logOut() [TestNG] PASSED CONFIGURATION: "Command line test eca2ebbd-c83a-4609-84f0-bb52a16b1afc" - @AfterMethod Tests.LoginTest.logOut() finished in 827 ms [Invoker 1508395126] Keeping method LoginTest.logOut()[pri:0, instance:Tests.LoginTest@15975490] for class [TestClass name=class Tests.LoginTest] ```

Due to Google Play Protect Warning selenium WebDriver failed to start the application. https://drive.corp.amazon.com/documents/zinnak@/Screen%20Shot%202022-05-05%20at%208.13.09%20PM.png Error: `selenium.common.exceptions.WebDriverException: Message: An unknown server-side error occurred while processing the command. Original error: Cannot start the 'com.ambient.escapes.mobile.dev' application. ` Issue is NOT observed on Samsung devices.

Trying to deploy a basic webdriver.io + nodejs test on devicefarm but always for ios device test cases are getting stuck at Testcase is failing: job arn: arn:aws:devicefarm:us-west-2:612756076442:job:02bd6c95-640d-43b3-82eb-6f618777ac73/1a6364f3-7528-44b1-afa1-d6c2dc51d881/00000 ``` 2022-05-04T22:40:58.353Z ERROR @wdio/runner: Error: Failed to create session. [0-0] Unable to connect to "http://localhost:4723/", make sure browser driver is running on that address. [0-0] If you use services like chromedriver see initialiseServices logs above or in wdio.log file as the service might had problems to start the driver. [0-0] at startWebDriverSession (/private/tmp/scratchY4h2F6.scratch/test-packagex5aknf/node_modules/integration/node_modules/webdriver/build/utils.js:72:15) [0-0] at process.processTicksAndRejections (node:internal/process/task_queues:95:5) [0-0] at async Function.newSession (/private/tmp/scratchY4h2F6.scratch/test-packagex5aknf/node_modules/integration/node_modules/webdriver/build/index.js:46:45) [0-0] at async remote (/private/tmp/scratchY4h2F6.scratch/test-packagex5aknf/node_modules/integration/node_modules/webdriverio/build/index.js:77:22) [0-0] at async Runner._startSession (/private/tmp/scratchY4h2F6.scratch/test-packagex5aknf/node_modules/integration/node_modules/@wdio/runner/build/index.js:223:56) ``` and for android testcases is asking for node version greater than 12 (after adding nvm install 18.1.0 in yaml)

I'm running 50 tests on parallel on Desktop browser using Nightwatch JS framework. Not all tests are starting and I keep getting error `Failed to connect to Selenium Server on testgrid-devicefarm.us-west-2.amazonaws.com with port 443` After further investigation I found this also is sent out ``` 500 POST https://testgrid-devicefarm.us-west-2.amazonaws.com/xxx=/wd/hub/session (1534ms) { value: { error: 'session not created' message: 'Cannot exceed 50 concurrent sessions. Currently active=48, pending=2, failed=0' } } Failed to connect to Selenium Server on testgrid-devicefarm.us-west-2.amazonaws.com with port 443 ``` Is this problem with Nightwatch or DeviceFarm ? What makes the test enter pending state ? How does AWS Device Farm selenium grid is not handling those requests ?

On reviewing the Device Farm, I understood we need to upload the apk file and test suite to AWS Farm and then the tests will be triggered in the devices from the device pool. Instead of manually uploading the test files and apks to farm, is there an option were the user can just trigger the tests from his local. And it will start running in AWS farm and after execution the results will be shared to user?

Hi there, I'm making attempts to run `CodeceptJS Appium` test suites for iOS and Android on Device Farm. Everything will go well, but an issue occurs when I turn on using `ResembleHelper` as following: ``` Could not load helper ResembleHelper from module 'codeceptjs-resemblehelper': dlopen(/private/tmp/scratch_01gRm.scratch/test-packagegv_xYF/node_modules/codeceptjs-dvf-poc/node_modules/resemblejs/node_modules/canvas/build/Release/canvas.node, 1): Library not loaded: /usr/local/opt/pango/lib/libpangocairo-1.0.0.dylib Referenced from: /private/tmp/scratch_01gRm.scratch/test-packagegv_xYF/node_modules/codeceptjs-dvf-poc/node_modules/resemblejs/node_modules/canvas/build/Release/canvas.node Reason: image not found Error: dlopen(/private/tmp/scratch_01gRm.scratch/test-packagegv_xYF/node_modules/codeceptjs-dvf-poc/node_modules/resemblejs/node_modules/canvas/build/Release/canvas.node, 1): Library not loaded: /usr/local/opt/pango/lib/libpangocairo-1.0.0.dylib ``` Is there any way to resolve it to make ResembleHelper usable on Device Farm? install missing packages or pull them from somewhere,... Thanks.

Suppose I have 50 tests in my suite and 5 devices in my device pool. Instead of running all 50 tests on each device, is it possible to run 10 tests on each device? I want to reduce the run time of our test suite.

I am trying to run Appium Python tests for Android devices. However, my test setup fails with the following error: 'ascii' codec can't decode byte 0xcc in position 58: ordinal not in range(128). My tests run locally just fine, and from the limited information in the logs here I'm unable to decipher where the problem lies. What steps can I take to troubleshoot this issue? Entirety of test setup log is below: ``` [{"attachment_id": null, "parent_id": null, "timestamp": "2022-04-12T23:02:46.018578Z", "level": "Info", "pid": 3106, "data": "Starting Setup Suite", "source": "Harness", "tag": null, "subtype": "Text", "tid": 3111, "type": "Message", "id": 13}, {"attachment_id": null, "parent_id": null, "timestamp": "2022-04-12T23:02:46.019150Z", "level": "Info", "pid": 3106, "data": "Starting Setup Test", "source": "Harness", "tag": null, "subtype": "Text", "tid": 3111, "type": "Message", "id": 15}, {"attachment_id": null, "parent_id": null, "timestamp": "2022-04-12T23:02:48.555726Z", "level": "Debug", "pid": 31496, "data": "Binder ioctl to enable oneway spam detection failed: Invalid argument", "source": "Device", "tag": "ProcessState", "subtype": "Text", "tid": 31496, "type": "Message", "id": 16}, {"attachment_id": null, "parent_id": null, "timestamp": "2022-04-12T23:02:52.875809Z", "level": "Errored", "pid": 3106, "data": "'ascii' codec can't decode byte 0xcc in position 58: ordinal not in range(128)", "source": "Harness", "tag": null, "subtype": "Text", "tid": 3127, "type": "Message", "id": 17}, {"attachment_id": null, "parent_id": null, "timestamp": "2022-04-12T23:02:53.334113Z", "level": "Errored", "pid": 3106, "data": "'ascii' codec can't decode byte 0xcc in position 58: ordinal not in range(128)", "source": "Harness", "tag": null, "subtype": "Text", "tid": 3111, "type": "Message", "id": 22}, {"attachment_id": null, "parent_id": null, "timestamp": "2022-04-12T23:02:53.334757Z", "level": "Info", "pid": 3106, "data": "Finished Setup Test", "source": "Harness", "tag": null, "subtype": "Text", "tid": 3111, "type": "Message", "id": 24}, {"attachment_id": null, "parent_id": null, "timestamp": "2022-04-12T23:02:53.336363Z", "level": "Info", "pid": 3106, "data": "Finished Setup Suite", "source": "Harness", "tag": null, "subtype": "Text", "tid": 3111, "type": "Message", "id": 27}] ```

Is there a way to set the device timezone to a particular time zone, America/Los Angeles (PDT) On Android? on iOS?

I am using Appium and NodeJS in a custom test environment and am finding that when a test run fails it gets wrongly reported as a pass. According to [the docs for Custom Test Environments](https://docs.aws.amazon.com/devicefarm/latest/developerguide/custom-test-environments.html), "If any command in the test phase fails, the test is marked as failed.". But this doesn't seem to be true. My test command is `node index.js $DEVICEFARM_APP_PATH` which installs the app on the device etc etc, and when the test fails that node process will exit with code 1, i.e. a fail (any non-zero process exit code is a failure, right?). But it gets reported in the AWS console as a pass. Is there some other way of reporting test fails that I've completely missed?

I want to know if there is data on DeviceFarm local files that could be used for web testing. If so, how can I access this data or log the path of the local files into the terminal. If not, is there's a way that I can use to upload files into the website during testing other than using S3 Bucket? Thank you in advance.

https://developer.android.com/training/safetynet/attestation

Hi Team, I'm currently trying to execute Appium automated test scenarios on AWS device farm. There are no errors in the log, but no test scenario executed. I have included the testng.xml file which refers to the TestRunner class. ``` [DEVICEFARM] Setting up your device. This usually takes 2-3 minutes. [DEVICEFARM] ########### Start executing testspec ########### [DEVICEFARM] ########### Entering phase install ########### [DeviceFarm] export APPIUM_VERSION=1.14.2 [DeviceFarm] avm $APPIUM_VERSION /usr/bin/avm: line 261: appium: command not found  exists :  [DeviceFarm] ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js [DEVICEFARM] ########### Entering phase pre_test ########### [DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TESTNG_JAR [DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/* [DeviceFarm] export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/dependency-jars/* [DeviceFarm] echo "Start appium server" Start appium server [DeviceFarm] appium --log-timestamp --default-capabilities "{\"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\", \"app\":\"$DEVICEFARM_APP_PATH\", \"udid\":\"$DEVICEFARM_DEVICE_UDID\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\", \"chromedriverExecutable\":\"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE\"}" >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 & [DeviceFarm] start_appium_timeout=0; while [ true ]; do if [ $start_appium_timeout -gt 60 ]; then echo "appium server never started in 60 seconds. Exiting"; exit 1; fi; grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1; if [ $? -eq 0 ]; then echo "Appium REST http interface listener started on 0.0.0.0:4723"; break; else echo "Waiting for appium server to start. Sleeping for 1 second"; sleep 1; start_appium_timeout=$((start_appium_timeout+1)); fi; done; Waiting for appium server to start. Sleeping for 1 second Waiting for appium server to start. Sleeping for 1 second Waiting for appium server to start. Sleeping for 1 second Waiting for appium server to start. Sleeping for 1 second Appium REST http interface listener started on 0.0.0.0:4723 [DEVICEFARM] ########### Entering phase test ########### [DeviceFarm] echo "Navigate to test package directory" Navigate to test package directory [DeviceFarm] cd $DEVICEFARM_TEST_PACKAGE_PATH [DeviceFarm] echo "Start Appium JUnit test" Start Appium JUnit test [DeviceFarm] java -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH org.testng.TestNG -junit -testjar *-tests.jar -d $DEVICEFARM_LOG_DIR/test-output [TestNG] Running: /tmp/scratch5Hh0UX.scratch/test-packagezOobaa/testng.xml =============================================== Regression Total tests run: 0, Failures: 0, Skips: 0 =============================================== [DEVICEFARM] ########### Entering phase post_test ########### [DEVICEFARM] ########### Finish executing testspec ########### [DEVICEFARM] ########### Setting upload permissions ########### [DEVICEFARM] Tearing down your device. Your tests report will come shortly. ``` https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-2#/mobile/projects/679dd7ea-c057-48a9-9fa6-cb581cf8133a/runs/716ddf89-b27a-4310-aecf-5eda2f2d7ac2/jobs/00000 Appreciate your help on this issue. Regards, Wathsala.

Unable to start Appium server when selecting Appium 1.22.0 in testspec.yml for Appium python tests error : **appium server never started in 60 seconds. Exiting**

Hi, I'm able to run cucumber specs (NodeJS & appium) successfully in aws device farm and framework reports generate in host machine file path: /private/tmp/scratchcd0WWN.scratch/test-packageXL7CdE/reports/html/index.html, how can I copy reports from this path location to customer artifact folder. I have included -$DEVICEFARM_LOG_DIR and $WORKING_DIRECTORY in .yml file. please help.!

Hello Guys, I am trying to run the appium automated tests with device farm. However, my tests are not running and failing with the below error. ``` [TestNG] RUNNING: Suite: "Command line test 41826786-55ed-4502-bf04-08c38bd76f95" containing "17" Tests (config: null) [Invoker 254801937] Keeping method AndroidBaseTest.setUpAndroidApp(org.testng.ITestContext)[pri:0, instance:mobile.androidapp.androidnativedemo.ApiDemoTestAndroid@32d2fa64] for class [TestClass name=class mobile.androidapp.androidnativedemo.ApiDemoTestAndroid] [Invoker 254801937] Invoking @BeforeClass AndroidBaseTest.setUpAndroidApp(org.testng.ITestContext)[pri:0, instance:mobile.androidapp.androidnativedemo.ApiDemoTestAndroid@32d2fa64] [TestNG] INVOKING CONFIGURATION: "Command line test 41826786-55ed-4502-bf04-08c38bd76f95" - @BeforeClass mobile.androidapp.common.AndroidBaseTest.setUpAndroidApp(org.testng.ITestContext)(value(s): org.testng.TestRunner@71c3b41) Failed to invoke configuration method mobile.androidapp.common.AndroidBaseTest.setUpAndroidApp:The main Appium script does not exist at '/usr/lib/node_modules/appium/build/lib/main.js' [TestNG] FAILED CONFIGURATION: "Command line test 41826786-55ed-4502-bf04-08c38bd76f95" - @BeforeClass mobile.androidapp.common.AndroidBaseTest.setUpAndroidApp(org.testng.ITestContext)(value(s): org.testng.TestRunner@71c3b41) finished in 0 ms [TestNG] io.appium.java_client.service.local.InvalidServerInstanceException: The main Appium script does not exist at '/usr/lib/node_modules/appium/build/lib/main.js' ``` Custom environment yaml file: ``` version: 0.1 phases: install: commands: - export APPIUM_VERSION=1.18.0 - avm $APPIUM_VERSION - ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js pre_test: commands: - export CLASSPATH=$CLASSPATH:$DEVICEFARM_TESTNG_JAR - export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/* - export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/dependency-jars/* - echo "Start appium server" - >- appium --log-timestamp --log-level debug:error --default-capabilities "{\"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\", \"app\":\"$DEVICEFARM_APP_PATH\", \"udid\":\"$DEVICEFARM_DEVICE_UDID\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\", \"chromedriverExecutable\":\"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE\"}" >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 & - >- start_appium_timeout=0; while [ true ]; do if [ $start_appium_timeout -gt 60 ]; then echo "appium server never started in 60 seconds. Exiting"; exit 1; fi; grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1; if [ $? -eq 0 ]; then echo "Appium REST http interface listener started on 0.0.0.0:4723"; break; else echo "Waiting for appium server to start. Sleeping for 1 second"; sleep 1; start_appium_timeout=$((start_appium_timeout+1)); fi; done; test: commands: - echo "Navigate to test package directory" - cd $DEVICEFARM_TEST_PACKAGE_PATH - echo "Start Appium TestNG test" - ls -ltr - which appium - echo $JAVA_HOME - which node - node -v - appium -v - java -Dappium.screenshots.dir=$DEVICEFARM_SCREENSHOT_PATH org.testng.TestNG -testjar *-tests.jar -d $DEVICEFARM_LOG_DIR/test-output -verbose 10 post_test: commands: artifacts: # By default, Device Farm will collect your artifacts from following directories - $DEVICEFARM_LOG_DIR ``` https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/mobile/projects/e7d85810-369a-443b-ac29-86ab38065b81/runs/2277ddb7-82ce-4aa5-8d40-3454f1baa009/jobs/00000 Could someone help me ? Regards, Sekar

Hello Guys, I am trying to run the appium Test NG automated tests using teng.xml file with device farm. However, my tests are not running after unzipping the tests.jar file. There is no error in the log file, so I am unable to find out the reason. Custom environment yaml file: ``` version: 0.1 phases: install: commands: # This test execution environment uses Appium version 1.9.1 by default, however we enable you to change it using the Appium version manager (avm). An - export APPIUM_VERSION=1.18.0 - avm $APPIUM_VERSION - ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js # The pre-test phase includes commands that setup your test environment. pre_test: commands: # Setup environment variables for java - export CLASSPATH=$CLASSPATH:$DEVICEFARM_TESTNG_JAR - export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/* - export CLASSPATH=$CLASSPATH:$DEVICEFARM_TEST_PACKAGE_PATH/dependency-jars/* - echo "Start appium server" - >- appium --log-timestamp --default-capabilities "{\"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\", \"app\":\"$DEVICEFARM_APP_PATH\", \"udid\":\"$DEVICEFARM_DEVICE_UDID\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\", \"chromedriverExecutable\":\"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE\"}" >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 & - >- start_appium_timeout=0; while [ true ]; do if [ $start_appium_timeout -gt 60 ]; then echo "appium server never started in 60 seconds. Exiting"; exit 1; fi; grep -i "Appium REST http interface listener started on 0.0.0.0:4723" $DEVICEFARM_LOG_DIR/appiumlog.txt >> /dev/null 2>&1; if [ $? -eq 0 ]; then echo "Appium REST http interface listener started on 0.0.0.0:4723"; break; else echo "Waiting for appium server to start. Sleeping for 1 second"; sleep 1; start_appium_timeout=$((start_appium_timeout+1)); fi; done; test: commands: # Your test package is downloaded in $DEVICEFARM_TEST_PACKAGE_PATH so we first change directory to that path. - echo "Navigate to test package directory" - cd $DEVICEFARM_TEST_PACKAGE_PATH # 1. Your testng.xml is inside the test jar after packaging step. Unzip it using commands: - echo "Unzipping TestNG tests jar" - unzip *-tests.jar # 2. Run your TestSuite by specifying the unzipped testng.xml using commands: - ls -ltr - which appium - appium -v - pwd - node -v - appium -v - which node - echo $DEVICEFARM_LOG_DIR - echo "Start Appium TestNG test" - java org.testng.TestNG -d $DEVICEFARM_LOG_DIR/test-output testng_android.xml ``` Error Log: ``` [DeviceFarm] echo "Start Appium TestNG test" Start Appium TestNG test [DeviceFarm] java org.testng.TestNG -d $DEVICEFARM_LOG_DIR/test-output testng_android.xml log4j:WARN No appenders could be found for logger (mobile.common.filereader.ConfigFileReaderUtils). log4j:WARN Please initialize the log4j system properly. log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info. [TestNG] Running: /tmp/scratchDVpVKr.scratch/test-packagemJj_ph/testng_android.xml =============================================== Android Regression Tests Total tests run: 2, Failures: 0, Skips: 2 Configuration Failures: 2, Skips: 1 =============================================== ``` https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/mobile/projects/e7d85810-369a-443b-ac29-86ab38065b81/runs/6d8ad0d2-394c-4de5-898a-eed8e816745c/jobs/00000 Could someone please help me out ? Regards, Sekar

Hi Team, I really like the idea of AWS Device Farm. Is it the same as SauceLabs? I have few questions: 1. Does AWS Device Farm supports Tricentis? 2. If not, does it provide any support tool to automate the mobile apps test cases? Tricentis is a no-code tool to automate the test cases. 3. Does AWS Device Farm allows to uploaded shielded iOS and Android apps? Br, Asmita

Hi, I'm getting the following error every time I try to run my Appium JS tests on iOS devices in AWS Device Farm: **WARN webdriver: Request failed with status 500 due to An unknown server-side error occurred while processing the command. Original error: Failed to receive any data within the timeout: 5000** For the following 2 attempts I get: **ERROR webdriver: RequestError: read ECONNRESET [0-0] at ClientRequest.<anonymous> (/private/tmp/scratchmnNKi2.scratch/test-packagecg02Ek/node_modules/webdriverio-tests/node_modules/got/dist/source/core/index.js:962:111) ** I tried with different Appium versions (1.14.2, 1.18.0, 1.19.0, even installing the latest 1.22.2 manually) as well as various iOS devices. Nothing helps nor changes the outcome. The app .ipa file is valid as I checked it earlier in manual connection sessions and it gets installed every time. So looks like some kind of a connection issue. The same tests work locally. My current desired capabilities look like this (although I also tried to run the tests *without any and got the same result*): ` { platformName: 'iOS', maxInstances: 1, 'appium:automationName': 'XCUITest', 'appium:noReset': true, 'appium:newCommandTimeout': 240 }` Shared capabilities look like this: exports.config = { // // ==================== // Runner Configuration // ==================== runner: 'local', path: '/wd/hub', port: 4723, specs: ['./specs/**/*.js'], // Patterns to exclude. exclude: [ // 'path/to/excluded/files' ], maxInstances: 1, logLevel: 'info', bail: 0, baseUrl: 'localhost', waitforTimeout: 20000, connectionRetryTimeout: 120000, connectionRetryCount: 3, framework: 'mocha', reporters: ['spec'], mochaOpts: { ui: 'bdd', timeout: 200000 //retries: 2 }, afterTest: async function (test, context, { error, result, duration, passed, retries }) { // take a screenshot anytime a test fails and throws an error if (!passed) { const testName = test.title.toString().split(' ').join('-') await driver.saveScreenshot(`./Failure_Screenshot_${testName}.png`) } } } Any help would be appreaciated as this error is not descriptive at all.

Hi, I am trying to change the location of the mobile device in the AWS device farm by providing correct latitude and longitude but it's now working. Can you please guide me?

The app I would like to test executes Android background services which perform various network tests to various sites. These results are reported to our backend for analysis. The UI is negligible, logging a terse test summary as tests occur. I started a session on a device, uploaded our app & set the appropriate permissions. The UI indicates tests are beginning. However, the background services do not appear to be executing. Is the AWS Device Farm intended to support UI interactions but NOT background service execution? Is there some setting in the console that would enable background service execution? Thanks.

Hello, I want to know if is possible to test an Android work profile provisioning in Device Farm with a private device, I guess in a public device is not possible because they have a profile of Amazon and I've tried. Can you help me confirming if this is possible, mean if with one of this kind (private or public devices) is it possible, and if is possible, What steps I have to follow?

I'm trying to sendKeys to a file handler in chrome and am getting: ` org.openqa.selenium.UnsupportedCommandException: unknown command: unknown command: session/1b651f6fc5776ba353569d3b7e423552/se/file` I am using the LocalFileDetector: driver.fileDetector = LocalFileDetector() This works locally as well as on other testing platforms. Is this not supported in Browser Testing(Device Farm)? Command: [1b651f6fc5776ba353569d3b7e423552, uploadFile {file=UEsDBBQACAgIAFqqTlQAAAAAAAAA .......

Hello everyone, I'm getting this error to try run ios tests on device farm: DVTErrorPresenter: Unable to load simulator devices in Device Farm Used tools: Appium version: 1.22.0 ios version: 15.x Link of run: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-2#/mobile/projects/a5325ea5-f090-4ac3-aabb-dcec96a31cd4/runs/7aba46fc-69db-4e2f-8170-93cb0115639d/jobs/00000 Logs: ``` [DeviceFarm] export TEST_PLATFORM=ios [DeviceFarm] export EXECUTION_ENVIRONMENT=AWS [DeviceFarm] export SLACK_SOS_FILE=/tmp/slackMSG.tmp [DeviceFarm] rm -f $SLACK_SOS_FILE [DeviceFarm] echo "Start Appium TestNG test" Start Appium TestNG test [DeviceFarm] java -Djdk.tls.client.protocols=TLSv1.2 -javaagent:dependency-jars/aspectjweaver-1.9.5.jar -Dallure.results.directory=/tmp/allure-results org.testng.TestNG -d $DEVICEFARM_LOG_DIR/test-output testng.xml WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/private/tmp/scratchXJtgqt.scratch/test-packagebB21R6/dependency-jars/groovy-2.5.8.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int) WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1 WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release [INFO ] ***oo try to write fileName: /tmp/slackMSG.tmp text: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/a5325ea5-f090-4ac3-aabb-dcec96a31cd4/runs/7aba46fc-69db-4e2f-8170-93cb0115639d [com.mach.core.util.FileUtil.writeFile(FileUtil.java:44)] [13:26:27,817] [0] [INFO ] ***oo capabilities from java: Capabilities {appPushTimeout: 1000, automationName: XCUITest, deviceName: 00008110-001A75102604801E, newCommandTimeout: 420, platformName: iOS, platformVersion: 15.1, updatedWDABundleId: com.facebook.WebDriverAgent..., useNewWDA: true} [com.mach.core.config.driver.AppiumDriverFactory.getIOSCapabilities(AppiumDriverFactory.java:165)] [13:26:27,963] [146] [ERROR] ***** check the capabilities, e: [com.mach.core.config.driver.AppiumDriverFactory.getIOSDriver(AppiumDriverFactory.java:122)] [13:27:20,945] [53128] org.openqa.selenium.SessionNotCreatedException: Unable to create a new remote session. Please check the server log for more details. Original error: An unknown server-side error occurred while processing the command. Original error: Unable to launch WebDriverAgent because of xcodebuild failure: xcodebuild failed with code 65 xcodebuild error message: 2022-02-02 13:27:10.270 xcodebuild[1191:8716] DVTErrorPresenter: Unable to load simulator devices. Domain: DVTCoreSimulatorAdditionsErrorDomain Code: 3 Failure Reason: The version of the CoreSimulator framework installed on this Mac is out-of-date and not supported by this version of Xcode. Recovery Suggestion: Please ensure that you have installed all available updates to your Mac's software, and that you are running the most recent version of Xcode supported by macOS. -- CoreSimulator is out of date. Current version (757.5.0) is older than build version (776.3.0). Domain: DVTCoreSimulatorAdditionsErrorDomain Code: 3 -- 2022-02-02 13:27:10.270 xcodebuild[1191:8716] iOSSimulator: [SimServiceContext sharedServiceContextForDeveloperDir:error:] returned nil (Error Domain=DVTCoreSimulatorAdditionsErrorDomain Code=3 "CoreSimulator is out of date. Current version (757.5.0) is older than build version (776.3.0)." UserInfo={NSLocalizedDescription=CoreSimulator is out of date. Current version (757.5.0) is older than build version (776.3.0).}). Simulator device support disabled. Command line invocation: /Applications/Xcode.app/Contents/Developer/usr/bin/xcodebuild test-without-building -project /usr/local/avm/versions/1.22.0/node_modules/appium/node_modules/appium-webdriveragent/WebDriverAgent.xcodeproj -scheme WebDriverAgentRunner -derivedDataPath /tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL -destination id=00008110001A75102604801E IPHONEOS_DEPLOYMENT_TARGET=15.1 GCC_TREAT_WARNINGS_AS_ERRORS=0 COMPILER_INDEX_STORE_ENABLE=NO User defaults from command line: IDEDerivedDataPathOverride = /tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL IDEPackageSupportUseBuiltinSCM = YES Build settings from command line: COMPILER_INDEX_STORE_ENABLE = NO GCC_TREAT_WARNINGS_AS_ERRORS = 0 IPHONEOS_DEPLOYMENT_TARGET = 15.1 2022-02-02 13:27:12.088 xcodebuild[1191:8717] DVTAssertions: Warning in /Library/Caches/com.apple.xbs/Sources/DVTiOSFrameworks/DVTiOSFrameworks-19114/DTDeviceKitBase/DTDKRemoteDeviceData.m:373 Details: (null) deviceType from 00008110001A75102604801E was NULL when -platform called. Object: <DTDKMobileDeviceToken: 0x7fbc90f8c120> Method: -platform Thread: <NSThread: 0x7fbc8ec124e0>{number = 3, name = (null)} Please file a bug at https://feedbackassistant.apple.com with this warning message and any useful information you can provide. 2022-02-02 16:27:19.723982-0500 WebDriverAgentRunner-Runner[417:15887] Running tests... 2022-02-02 16:27:19.726461-0500 WebDriverAgentRunner-Runner[417:15887] Unable to load configuration data from specified path /; error: The file “System” couldn’t be opened. 2022-02-02 16:27:19.726518-0500 WebDriverAgentRunner-Runner[417:15887] XCTRunner Arguments: ( "/var/containers/Bundle/Application/3B1C00BC-8B46-437E-923A-C3A20F6DA262/WebDriverAgentRunner-Runner.app/WebDriverAgentRunner-Runner" ) 2022-02-02 16:27:19.726621-0500 WebDriverAgentRunner-Runner[417:15887] XCTRunner Environment: { "CA_ASSERT_MAIN_THREAD_TRANSACTIONS" = 0; "CA_DEBUG_TRANSACTIONS" = 0; "CFFIXED_USER_HOME" = "/private/var/mobile/Containers/Data/Application/3CE450E6-4DEF-46F3-BB9C-ACB104BA7E97"; HOME = "/private/var/mobile/Containers/Data/Application/3CE450E6-4DEF-46F3-BB9C-ACB104BA7E97"; LOGNAME = mobile; "MJPEG_SERVER_PORT" = ""; "MTC_CRASH_ON_REPORT" = 1; NSUnbufferedIO = YES; "OS_ACTIVITY_DT_MODE" = YES; PATH = "/usr/bin:/bin:/usr/sbin:/sbin"; "RUN_DESTINATION_DEVICE_ECID" = 7447061612232734; "RUN_DESTINATION_DEVICE_NAME" = PDX640550096; "RUN_DESTINATION_DEVICE_PLATFORM_IDENTIFIER" = "com.apple.platform.iphoneos"; "RUN_DESTINATION_DEVICE_UDID" = 00008110001A75102604801E; SHELL = "/bin/sh"; "SQLITE_ENABLE_THREAD_ASSERTIONS" = 1; TMPDIR = "/private/var/mobile/Containers/Data/Application/3CE450E6-4DEF-46F3-BB9C-ACB104BA7E97/tmp/"; "UPGRADE_TIMESTAMP" = 1643837138604; USER = mobile; "USE_PORT" = 8100; "WDA_PRODUCT_BUNDLE_IDENTIFIER" = "com.facebook.WebDriverAgentRunner"; XCTestBundlePath = "PlugIns/WebDriverAgentRunner.xctest"; XCTestConfigurationFilePath = ""; XCTestSessionIdentifier = "88DCC108-B122-4D1C-84E4-DA839C9021CD"; "XPC_FLAGS" = 0x0; "XPC_SERVICE_NAME" = "UIKitApplication:com.apple.test.WebDriverAgentRunner-Runner[9ca9][rb-legacy]"; "__CF_USER_TEXT_ENCODING" = "0x1F5:0:0"; } 2022-02-02 13:27:19.643 xcodebuild[1191:8639] [MT] IDETestOperationsObserverDebug: 6.524 elapsed -- Testing started completed. 2022-02-02 13:27:19.643 xcodebuild[1191:8639] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start 2022-02-02 13:27:19.643 xcodebuild[1191:8639] [MT] IDETestOperationsObserverDebug: 6.524 sec, +6.524 sec -- end Test session results, code coverage, and logs: /tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL/Logs/Test/Test-WebDriverAgentRunner-2022.02.02_13-27-13--0800.xcresult Testing failed: WebDriverAgentRunner: WebDriverAgentRunner-Runner (417) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. (Underlying Error: The test runner exited with code 1 before checking in. If you believe this error represents a bug, please attach the result bundle at /tmp/scratchXJtgqt.scratch/DerivedDatakuxIoL/Logs/Test/Test-WebDriverAgentRunner-2022.02.02_13-27-13--0800.xcresult)) ** TEST EXECUTE FAILED ** Testing started. Make sure you follow the tutorial at https://github.com/appium/appium-xcuitest-driver/blob/master/docs/real-device-config.md. Try to remove the WebDriverAgentRunner application from the device if it is installed and reboot the device. Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03' System info: host: 'ip-192-168-10-5.us-west-2.compute.internal', ip: 'fe80:0:0:0:14c0:e6ff:c23e:5cb3%en0', os.name: 'Mac OS X', os.arch: 'x86_64', os.version: '10.16', java.version: '12.0.2' Driver info: driver.version: IOSDriver ... ```

Problem: my tests are not running and eventually get terminated. While I see both the app and WDA getting installed in the video, the app will never get launched. Testspec output does not reveal any meaningful info, only shows "Terminated: 15". Reviewing the syslog output, I'm able to see a couple of issues with WDA: ``` WebDriverAgentRunner-Runner(WebDriverAgentLib)[416] <Notice>: Enqueue Failure: Application 'test.bundle.id.here' has not loaded accessibility /Users/saaandr/Downloads/WebDriverAgent-2.20.8/WebDriverAgentRunner/UITestingUITests.m 38 1 ``` and ``` WebDriverAgentRunner-Runner(WebDriverAgentLib)[416] <Notice>: Enqueue Failure: Failed to get automation session for test.bundle.id.here:0: Timed out while requesting automation session for test.bundle.id.here (pid:0) ``` [ARN run](https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/mobile/projects/c248fb2e-1df2-4c2c-8499-5bf7523ce683/runs/4e9ab72e-c9d7-4081-ad7d-9341c4d10eb2): arn:aws:devicefarm:us-west-2:223051165180:run:c248fb2e-1df2-4c2c-8499-5bf7523ce683/4e9ab72e-c9d7-4081-ad7d-9341c4d10eb2 As far as desired capabilities, I am not modifying anything from the default capabilities. Also, the testspec yml uses the latest testspec for iOS (v 7.0) as a starting point, modified only with a custom py.test command. This is the behavior that I see when I choose to run test on an iOS14.0, or 13.3.1 device. Executing the xcodebuild -version command in the install phase of the testspec shows that it is running Xcode 11.7, which was released back in Sept 2020. The test run eventually gets terminated by the WebDriverAgentRunner not being able to find the app running. But when I attempt to run on a device with iOS14.8 or 15.x, the setup fails within 2 minutes, since the app cannot be installed due to a resigning issue, similar to [this post](https://repost.aws/questions/QUy77blDSqQc-I6jnDZfvY5w/i-phone-application-not-installing-on-i-os-14-8#CO4MziupFYRJ6KpGTsnkiqmA). Worth mentioning, this same test runs with no problems on an Android device. Any help is appreciated.

We are testing an Android Cordova application in Device Farm using Appium. We use the hack from here to get the chromedriver working on AWS: https://forums.aws.amazon.com/thread.jspa?messageID=983382&#983382 . This has been working well until recently. We are now getting this error after switching to the webview context: `[0-0] 2022-01-14T16:27:50.188Z ERROR webdriver: Request failed with status 500 due to unknown error: An unknown server-side error occurred while processing the command. Original error: Process didn't end after 10000ms (cmd: '/opt/chromedriver/linux/65/chromedriver --url-base\=wd/hub --port\=8000 --adb-port\=5037 --verbose') [0-0] Error in "chat landscape can send chat messages in landscape" UnknownError: An unknown server-side error occurred while processing the command. Original error: Process didn't end after 10000ms (cmd: '/opt/chromedriver/linux/65/chromedriver --url-base\=wd/hub --port\=8000 --adb-port\=5037 --verbose')` Restarting the unchanged code that has been working before now keeps returning this error. The only thing that changed is that we bought a Device slot on Device Farm. Does anybody have any clues?

I am trying to install an application build on iOS 11 (XCode 13) on a remote session device. On an iPhone 11 (iOS 14.0) the app installs (but does not start properly, but that's most likely a separate issue). However, on iPhone 12 (iOS 14.8) the installation fails and I can see the following error in syslogs: ``` Jan 14 05:42:42 installd(MobileSystemServices)[253] <Notice>: 0x16b8cf000 +[MICodeSigningVerifier _validateSignatureAndCopyInfoForURL:withOptions:error:]: 77: Failed to verify code signature of /var/installd/Library/Caches/com.apple.mobile.installd.staging/temp.dJ1hU0/extracted/Payload/<redacted>.app : 0xe8008029 (The code signature version is no longer supported.) ``` The error may hint towards the signature having an old format (https://developer.apple.com/documentation/xcode/using-the-latest-code-signature-format) although the versions do not add up. Since Device Farm resigns the application with its own provisioning profile it would be helpful to know which version of macOS the resigning is running on (or if it's using the `--generate-entitlement-der` flag). Might be similar to https://repost.aws/questions/QUNcilMRiSQqKdWhXRUjkzyQ/application-verification-failed-with-code-0-xe-8008029-failed-to-verify-code-signature

I'm trying to test my application on iOS using Device Farm Remote Access. One of the features in test is sharing content to Instagram. Is there a way of installing Instagram on the device so I can test this feature?

We are trying to run End To End test on AWS Device Farm. Everyting work fine on Android. But we cannot make it work for IOS and we are close to give up. It's seems the ipa i provide cannot be installed on the real device, see the logs here : `Failed - Failed to install com.myapp.preprod - Install failed. Got error "ApplicationVerificationFailed" with code 0xe8008029: Failed to verify code signature of /var/installd/Library/Caches/com.apple.mobile.installd.staging/temp.Vk11Hw/extracted/Payload/EPPROD.app : 0xe8008029 (The code signature version is no longer supported.)` For some context it's an app build on react native with Expo, and build by Expo with XCODE 13.0.

Hi, I'm trying to run a small test for an Android app using Appium and TestNG on AWS device farm and I'm facing this problem. As recommendation from AWS, I just kept the desired capabilities empty like this and I could run the test successfully on my local machine ``` @BeforeSuite public void setUpAppium() throws MalformedURLException { final String URL_STRING = "http://127.0.0.1:4723/wd/hub"; URL url = new URL(URL_STRING); DesiredCapabilities capabilities = new DesiredCapabilities(); driver = new AndroidDriver<MobileElement>(url, capabilities); driver.manage().timeouts().implicitlyWait(35, TimeUnit.SECONDS); } ``` Then I followed all the steps in the instruction: uploaded the apk, zip file, etc. to AWS Device farm. And then when I ran the test on AWS, I got this error: > Cannot start the 'com.simplemobiletools.notes.pro' application. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting. Original error: Activity name '.com.simplemobiletools.notes.pro.activities.SplashActivity.Grey_black' used to start the app doesn't exist or cannot be launched As the error message, AWS tried to launch the activity ".com.simplemobiletools.notes.pro.activities.SplashActivity.Grey_black" which is kind of weird because when I used the command "adb shell dumpsys window | grep -E 'mCurrentFocus'" to check the app activity then I got this result "com.simplemobiletools.calculator.activities.MainActivity" and I don't know where AWS got that activity. So I guessed because of the wrong activity, appium couldn't start my app and the test couldn't be run on the device farm. I also checked with other apps and I saw the same problem. App activity that I used the adb shell command to check is always different to the app activity AWS got. AWS seems to tend to add the word "Splash" into the app activity and I don't know what it meant I tried this solution from this link "[https://forums.aws.amazon.com/thread.jspa?messageID=959277&tstart=0](https://forums.aws.amazon.com/thread.jspa?messageID=959277&tstart=0)" by adding appActivity and appWaitActivity to the yaml file ``` - echo "Start appium server" - >- appium --log-timestamp --default-capabilities "{\"deviceName\": \"$DEVICEFARM_DEVICE_NAME\", \"platformName\":\"$DEVICEFARM_DEVICE_PLATFORM_NAME\", \"app\":\"$DEVICEFARM_APP_PATH\", \"udid\":\"$DEVICEFARM_DEVICE_UDID\", \"platformVersion\":\"$DEVICEFARM_DEVICE_OS_VERSION\", \"chromedriverExecutable\":\"$DEVICEFARM_CHROMEDRIVER_EXECUTABLE\,\"appActivity\":\"com.simplemobiletools.notes.pro.activities.MainActivity\",\"appWaitActivity\":\"com.simplemobiletools.notes.pro.activities.MainActivity\"}" >> $DEVICEFARM_LOG_DIR/appiumlog.txt 2>&1 & - >- ``` but it didn't work either. Now I get another error "Waiting for appium server to start. Sleeping for 1 second". I tried to find any solution but couldn't find anything Please help me resolve this issue, I really appreciate your help!

Hi, I am facing a problem while using Testng + appium test type on aws devicefarm. Taking screenshot on iOS 15.0.2 devices, gives a screenshot exception. This code is working fine on iOS 14.x and 13.x devices. On line: ((TakesScreenshot) driver).getScreenshotAs(OutputType.FILE); [TestNG] org.openqa.selenium.remote.ScreenshotException: Screen shot has been taken I suspect this issue is because of prebuilt WDA version V6, as I am able to reproduce it locally. Please check https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/mobile/projects/ba8e59b9-76a8-40b2-9928-951bfdaf6e1b/runs/f7ea613e-bf68-4f74-9922-daef217949f4 Thanks

I'm creating a device pool with AWS CLI and want to exclude 2 models, but they keep showing in the created pool, any advice? Thanks Note: I've also tried excluding with MANUFACTURER, but it does not work either. These are the rules I'm using: ``` [ { "attribute": "AVAILABILITY", "operator": "EQUALS", "value": "\"HIGHLY_AVAILABLE\"" }, { "attribute": "PLATFORM", "operator": "EQUALS", "value": "\"ANDROID\"" }, { "attribute": "FORM_FACTOR", "operator": "EQUALS", "value": "\"PHONE\"" }, { "attribute": "OS_VERSION", "operator": "GREATER_THAN_OR_EQUALS", "value": "\"8.0.0\"" }, { "attribute": "OS_VERSION", "operator": "LESS_THAN_OR_EQUALS", "value": "\"11\"" }, { "attribute": "MODEL", "operator": "NOT_IN", "value": "[\"OnePlus One\",\"LG Stylo 5\"]" } ] ```

Can I use AWS Device Farm to test applications on PCs i.e. Windows PC across Internet explorer, Chrome Browser, Edge Etc? Can I use this on Apple PCs to test on Safari ? What other platforms / browsers are supported apart from the mobile platforms listed in the documentations. is the product SOC compliant ? if not are there any mitigating controls to put in place for using this.

We are currently running UI tests against our application in our AWS Account against Chrome, Edge and Firefox browsers using Selenium Grid running on ECS. We would like to also run tests against the Safari browser. As Safari can't run on Docker we can't use ECS. AWS Device Farm also does not currently support Safari. We are investigating other options to run tests against Safari such as using a Safari Selenium node running on EC2 (running MacOS). Does anyone have any experience with running tests against the Safari browser in AWS?

Hello, We're getting INSTALL_PARSE_FAILED_NO_CERTIFICATES error when we launch a test run with our AndroidSDK-30 app through the device farm. The error occur during the Setup phase no matter which kind of Android devices we select: Failed to install com.** - INSTALL_PARSE_FAILED_NO_CERTIFICATES: Scanning Failed.: No signature found in package of version 2 or newer for package com.** Any idea of what we could be doing wrong? For info, we don't have any issues to install the apk on our own devices. Thanks, Martin

According to DF documentation, "_Device Farm supports all Appium server versions 1.6.5 and above._" Appium 1.21 & 1.22 requires at least Node v12, but the current Node version in DF is set to v10. How do I update the version or set it to a specific version? I tried "n lts" in the yaml file, but got "mkdir: cannot create directory ‘/usr/local/n’: Permission denied". There are fixes in 1.21/1.22 that we need for mobile testing.

We are unable to access the clipboard when running auto on an Android S7 device and get the following message in logcat: "ClipboardService: Denying clipboard access to io.appium.uiautomator2.server, application is not in focus nor is it a system service for user 0" However, we are able to access the clipboard on the following AWS devices: S4 (Android 8.1) S6 (Android 9) and the following local devices not on AWS: S5e tablet (Android 11) S9 (Android 10) Android 11 emulator image This issue seems to be specific to AWS's S7 device. Below is an excerpt from logcat. 10-06 16:31:07.958 8131 8412 I appium : channel read: POST /wd/hub/session/8e1fdceb-781a-4d2c-bcd8-8cc9edff0fef/appium/device/get_clipboard 10-06 16:31:07.958 8131 8412 I appium : Get Clipboard command 10-06 16:31:07.962 1136 2189 E ClipboardService: Denying clipboard access to io.appium.uiautomator2.server, application is not in focus nor is it a system service for user 0 10-06 16:31:07.963 8131 8412 I appium : AppiumResponse: {"sessionId":"8e1fdceb-781a-4d2c-bcd8-8cc9edff0fef","value":""}

hello, I'm trying to set up iOS automation testing using XCTestUI & AWS device farm, but all tests are failing when are ran in device-farm: " container_create_or_lookup_app_group_path_by_app_group_identifier: client is not entitled " To me, it looks like XCTestUI bundle can't access (or find) app group container, but I can't figure out why. I double and triple checked AppGroups permissions but i'm not seeing anything wrong there. Same test are running ok in xcode emulator or on several real devices I have been using. So maybe there's something else I specifically need to be doing for aws device-farm? Any help would be greatly appreciated!

Hello people, I'm having a problem running tests on ios devices in device farm. I have a react-native project that builds both android and ios application, with appium tests for both of them. I'm using webdriverio + appium. The tests work locally, and on the android side I managed to make them work aswell in Device Farm. The problem: when running iOS tests, the webdriver agents cannot find the elements (e.g. login form) of my ios application. I keep getting "error: 'no such element' ". The application starts, and the webdriver calls seems correct, but at the end it cannot find a single element, and thus the tests fail. I've tried changing the capabilities, host, path, wda, appium/node versions and a lot else, but to no avail. Here is one of my recent run: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=eu-west-1#/projects/5ffc4bec-1370-4f5e-8823-4f5596a49b0d/runs/d67e226a-9526-4ddd-b0ea-627d092c7fd7 I would really appreciate some insights on this, I'm a bit out of ideas. Thanks for your time!

I have a pool definition for Android phones and I am getting Moto G7 Play devices moderately frequently of late and our app requires 64 bit devices and so fails to install. Need advice on how to avoid 32 bit devices.

The test durantion is about 1 minute when I create the test by AWS console, refer to https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/6c1de135-24c6-45d5-aaad-855d2395585b/runs/50367d4a-f297-436c-8b76-ded911849cb6/jobs/00000 But, the same test took more than 10 minutes when I create it by AmazonDeviceFarmClient, refer to https://us-west-2.console.aws.amazon.com/devicefarm/home#/projects/6c1de135-24c6-45d5-aaad-855d2395585b/runs/78f14d4b-f7e9-454a-9b21-40be1661ce75/jobs/00000 From the log I can see the test passed at 47s but somehow took 10 minutes to finish. Is there anything I did wrong? Thanks Harness 00:31.931 2506 Info Finished Setup Test Harness 00:31.943 2506 Info Finished Setup Suite Harness 00:31.944 2506 Info Starting Tests Suite Harness 00:31.945 2506 Info Starting Tests Harness 00:47.615 2506 Passed Tests passed Harness 05:51.741 2506 Info Finished Tests Harness 05:51.742 2506 Info Finished Tests Suite Harness 05:51.747 2506 Info Starting Teardown Suite Harness 05:51.748 2506 Info Starting Teardown Test Harness 11:17.347 2506 Passed Successful test lifecycle of Teardown Test Harness 11:17.347 2506 Info Finished Teardown Test Harness 11:17.348 2506 Info Finished Teardown Suite

Hello everybody. I'm using AWS Device Farm to run my tests. This is web to mobile testing for video calls, where both desktop pc and mobile phone must be used in one framework. I have a problem to connect to my remote machine in test while using iOS, but everything is fine with Android. I have both Android and iOS applications for mobile testing. The steps I do to run tests using AWS device farm: 1. We have a remote machine with open static IP address where I have a selenium-server-standalone started; 2. I put my project to AWS Device Farm -> Mobile device testing project -> Create a new project; 3. I fill out all the blanks, upload my application, select Java Appium TestNG, set configuration, create device pool and launch run; 4. Project goes to AWS Device Farm and while method 'setup' is invokes both the remote machine and selected mobile device and tests run; When I use my Android application, the test is able to connect to my remote machine and then invoke Android phone, then tests run in parallel on both web pc and mobile device. But when I try to do the same thing with iOS application, I have an unknown problem. I do the same steps with Android. So, the difference is only that it is iOS phone. And when I try to connect to my remote machine, the test skips. Without any log or comment...just skips and that's it. The way I connect to my machine from AWS Device farm mobile project: @BeforeTest public void setUp() { //web DesiredCapabilities dc = new DesiredCapabilities(); dc.setBrowserName("chrome"); dc.setPlatform(Platform.ANY); URL remoteHostURL = new URL("http://x.x.x.x:4444/wd/hub"); DRIVER_WEB.set(new RemoteWebDriver(remoteHostURL, dc)); //mobile capabilities = CapabilityManager.getAndroidCapabilities(); DRIVER_MOBILE.set(new AppiumDriver<MobileElement>(remoteAppiumHostURL, capabilities)); ... } What could cause this problem? Cos it works only for Android, but doesn't work for iOS. I'd be very thankful to get any help from you!

I'm working on adapting functional tests of a web to be able to use Device Farm as the device pool for testing in mobile devices. The tests are written in Java, using TestNG and Appium, and i have no problems launching them in desktop browsers locally or in my android phone through a local installation of Appium. I have adapted the pom.xml following the guidelines in <https://docs.aws.amazon.com/devicefarm/latest/developerguide/test-types-appium.html> , but i haven't found anything in the documentation stating how i should instantiate WebDriver. I have also reviewed and tested with the method to instantiate it as a RemoteWebDriver using the sdk like this: String myProjectARN = "arn:evicefarm:us-west-2:XXXXXXXXXX:testgrid-project:XXXXXXXX"; DeviceFarmClient client = DeviceFarmClient.builder() .region(Region.US_WEST_2) .build(); CreateTestGridUrlRequest request = CreateTestGridUrlRequest.builder() .expiresInSeconds(300) .projectArn(myProjectARN) .build(); CreateTestGridUrlResponse response = client.createTestGridUrl(request); URL testGridUrl = new URL(response.url()); WebDriver driver = new RemoteWebDriver(testGridUrl, DesiredCapabilities.chrome()); This works fine for testing in desktop environments, but if i try to adapt it to android environment, i get the following error: \[TestNG] software.amazon.awssdk.services.devicefarm.model.DeviceFarmException: User: arn:aws:sts::XXXXXXXXXXXXX:assumed-role/EC2DeviceHostRole/XXXXXXXXXXXXXXXX is not authorized to perform: devicefarm:CreateTestGridUrl on resource: arn:aws:devicefarm:us-west-2:XXXXXXXXXXXXX:project:XXXXXXXXXXXXXXXXXXXXXXXXX (Service: DeviceFarm, Status Code: 400, Request ID: XXXXXXXXXXXXXXXXXXXXXX, Extended Request ID: null) How should WebDriver be instantiated for mobile tests? My goal is to run this tests both in Android and iOS devices.

Hello AWS, I am running my scripts on AWS device farm(Automation testing) and I am using the java+testNG environment for automation. My requirement is, I need a "Cucumber.json" file that is generating after every successful execution of the script. I have tried out many things but unable to get this file. but the same thing it's generating locally. can you help me with this? sample Code:- ``` public class ReportHelper { private static final String REPORT_OUTPUT_DIRECTORY = System.getenv("DEVICEFARM_LOG_DIR"); public static void generateCucumberReport(String testfeature) { File reportOutputDirectory = new File(REPORT_OUTPUT_DIRECTORY); ArrayList<String> jsonFiles = new ArrayList<String>(); jsonFiles.add("tmp/cucumber.json"); String projectName = "ion-robot-app-automation"; Configuration configuration = new Configuration(reportOutputDirectory, projectName); configuration.addClassifications("Platform", "iOS"); ReportBuilder reportBuilder = new ReportBuilder(jsonFiles, configuration); try { reportBuilder.generateReports(); } catch (Exception ex) { System.out.println("report generation failed" + ex.getMessage()); } } ``` Thanks, Abhishek Garg Edited by: Tobe-AWS on Feb 22, 2021 5:03 PM

hi, Could you share sample custom mode settings script that install jdk1.8 and do the export with me? I tried this one but it didnt work. https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-2#/projects/e3de778f-4d0b-45a3-be78-fada79567b90/runs/1929a132-9351-4c63-a0b7-064db540379a/jobs/00000 Thanks, Edited by: sbuyuran on Jan 25, 2021 10:30 PM

Hi, I tired to test a simple app with xctest framework on Device Farm. It shows "Setup Test skipped: Not enough suites found to execute ". but the app did have some tests. The app source code: https://github.com/lloydsheng/DeviceFarmTest The job link: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/fe31921c-c291-46de-9c8f-620a112fe648/runs/43dc3d5a-4140-4be7-a627-98f3de086164/jobs/00000 The development environment: Xcode 12.2 macOS 10.15.7

Hello, I have a test suite that takes more than 150 minutes to complete both in android and ios, and I wanted to run my tests in parallel (running different tests from the same test suite in multiple devices) I bought two slots (one for android and one for ios), so would it be possible for me to implement test sharding? or is there any other strategy I'm not aware of? because a lot of my tests aren't run since they exceed the time limit of 150 minutes per device. Thank you very much Edited by: aguspeq on Jan 16, 2021 10:40 AM

For some reason, all of a sudden, runs are being skipped. Absolutely no mention within the run as to why. Is there a way to find out what is causing the run to be skipped? The same test file, spec file, and iOS app were working moments ago. It is a bit frustrating trying to find a cause when DF doesn't provide any information - but maybe I am not looking in the right place.

Hi, I am trying to run test on Android app with Appium and TestNG. At first I used empty desired capabilities but then I discovered that adding following caps helped to certain users on this forum: capabilities.setCapability("appPackage", "com.everfx"); capabilities.setCapability("appActivity", "com.zoontek.rnbootsplash.RNBootSplashActivity"); capabilities.setCapability("appWaitPackage", "com.everfx"); capabilities.setCapability("appWaitActivity", "com.zoontek.rnbootsplash.RNBootSplashActivity"); capabilities.setCapability("appWaitDuration", "50000"); It is interesting, because I thought that AWS Device Farm is overriding any capabilities that appium server is receiving. Unfortunately it didn't help and I am still received the same error: **Failed to invoke configuration method test.BaseTestClass.setUp:Unable to create a new remote session. Please check the server log for more details. Original error: An unknown server-side error occurred while processing the command. Original error: Cannot start the 'com.everfx' application. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting. Original error: 'com.zoontek.rnbootsplash.RNBootSplashActivity' or 'com.everfx.com.zoontek.rnbootsplash.RNBootSplashActivity' never started. Visit https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/activity-startup.md for troubleshooting** Then I decided to change "appWaitActivity" to receive ".MainActivity", as Appium might skip seeing splashscreen, however it didn't help either. Service was still expecting to see "com.zoontek.rnbootsplash.RNBootSplashActivity" which might mean that capabilities are not being overridden at all. So question is how to pass capabilities to aws device farm properly? And how to overcome the issue that tests are not seeing app?

Hi, I am working with AWS device farm for a while testing only one Mobile App and also using the Jenkins CI plugins. But at this moment I have the complexity of testing in a one test, two Mobile applications and interaction with a desktop browser (Must be desktop browser). I was thinking of using the desktop version of AWS device farm. This is the stage: 1. Install two Apps 2. Interactue App1 3. Interctue App2 4. Interactue Browser 5. Interactue App1 6. Interactue Browser Questions: - I can install two applications on device farm and use with jenkins plugin? - I can interact in a single mobile test and desktop browser in aws device farm? It would be very helpful if you have an example of something similar Thank you

I am unable to upload the apk on to a device on aws device farm, no error message is being displayed. I am having the same issue when running a test suite the set up passes but then tests fail, when I watched the video playback the app is not visible or installed. Is there also somewhere I could look for errors / help / logs to guide me to locate the issue. Edited by: QAtesterAWS on Aug 11, 2020 3:09 AM

I read through the previous posts around this subject and changed my code so that I pass $WORKING_DIRECTORY to my code and the tests execute and tell me that they created reports, etc in 2020-07-01 20:59:26 Messager \[main-1] \[INFO] \[SM-T860] RESULT #1: TEST \[SetupOnly - testSetupFlow] PASS \[screenshots=file:// /tmp/scratchTm8wqJ.scratch/test-packageJ9ckZb/tmp/customer_artifacts_working_directorytD0aTZ/1593636962238/3fda58af-0cf0-4bed-90a4-707eb8f2efdb/report.html | log=file:// /tmp/scratchTm8wqJ.scratch/test-packageJ9ckZb/tmp/customer_artifacts_working_directorytD0aTZ/1593636962238/3fda58af-0cf0-4bed-90a4-707eb8f2efdb/test.log]. URL should look like a file url for tmp/scratchTm8wqJ.scratch/test-packageJ9ckZb/tmp/customer_artifacts_working_directorytD0aTZ/1593636962238/3fda58af-0cf0-4bed-90a4-707eb8f2efdb/report.html I do not get the reports in the customer artifacts. I tried adding $WORKING_DIRECTORY in the artifacts section of the test spec and my test spec output shows: chmod: cannot access ‘/tmp/customer_artifacts_working_directorytD0aTZ/*’: Permission denied Help??? Edited by: hclmobiletests on Jul 1, 2020 2:09 PM edited the urls in the output pointing to the report.html and test.log so they don't get modified (add space between file: // and the directory). Edited by: hclmobiletests on Jul 1, 2020 2:11 PM another attempt to show the file path for the url

I am testing both an iOS and an Android native app with Appium/TestNG and my automation code is cross platform but only supports java8. I build my zip and run my tests successfully on Android, but on iOS, the java version that I am finding is java 12. I searched this forum and saw that previously someone had the opposite issue (needed java > 8) and their workaround was to install the jdk and do the exports. I have that approach working, but am wondering if there a way in the Test Spec to specify java8 (so I don't have to provide it)?

Hi 1)When tried to run app through appium testng shown following error on console -shown error "Activity never started" 2)But when i tried to launch app on my device through adb shell with package/activity name i am able to launch the app. adb shell am start -n <package/activity> 3)Appium - Testng able to launch app in my device and works fine 4) I have tried to run in device farm by not providing app activity capability in first run and with app activity in second run , but both the times it has failed Below are run's url: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/3c91870d-d21d-41a7-bf65-c051dd5ba700/runs/f3c0a151-5064-4004-92bb-33a22535a839 https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/3c91870d-d21d-41a7-bf65-c051dd5ba700/runs/1cb115c5-4150-48b3-b6d8-f45445d7d407 Please help. Edited by: VarunN on May 12, 2020 6:10 AM Edited by: VarunN on May 12, 2020 4:02 PM

Hi, I have created test for React Native application using Appium and WebdriverIO, it&#39;s working fine locally on android emulator and on real device, but it&#39;s not working on aws device farm. This tutorial <https://medium.com/jetclosing-engineering/react-native-device-testing-w-appium-node-and-aws-device-farm-295081129790> I followed to create test. **Issue is** 2020-05-06T14:01:27.388Z INFO webdriver: DATA { using: &#39;accessibility id&#39;, value: &#39;loginEmail&#39; } 2020-05-06T14:01:27.390Z INFO webdriver: COMMAND findElement("accessibility id", "app-root") 2020-05-06T14:01:27.393Z INFO webdriver: \[POST] http://127.0.0.1:4723/wd/hub/session/5a7350a3-e8c2-4c9e-b31a-347ea4defcdb/element 2020-05-06T14:01:27.393Z INFO webdriver: DATA { using: &#39;accessibility id&#39;, value: &#39;app-root&#39; } \[0-0] TypeError in "Simple App testing Valid Login Test" **$(...).setValue is not a function** the same issue is for waitForDisplayed **$(...).waitForDisplayed is not a function** **Device Farm Project Test URL**: <https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/df9dc8c9-3ddc-4822-85ef-1991208ae80e/runs/31e7f665-380e-4e21-bd17-8ff085474ecb> **Below is my package.json file** { "name": "tests", "version": "1.0.0", "description": "", "main": "index.js", "scripts": { "test": "npm install && npx wdio ./wdio.conf.local.js", "package": "npm install && npm-pack-zip" }, "keywords": \[], "author": "", "license": "ISC", "dependencies": { "@wdio/cli": "^6.1.2", "@wdio/local-runner": "^6.1.2", "@wdio/mocha-framework": "^6.1.0", "@wdio/spec-reporter": "^6.0.16", "@wdio/sync": "^6.1.0", "wdio-chromedriver-service": "^6.0.2", "chromedriver": "^81.0.0", "chai": "^4.2.0", "webdriverio": "^6.1.2" }, "devDependencies": { "npm-bundle": "^3.0.3", "npm-pack-zip": "^1.2.7" }, "bundledDependencies": \[ "@wdio/cli", "@wdio/mocha-framework", "@wdio/local-runner", "@wdio/spec-reporter", "@wdio/sync", "webdriverio", "chai", "chromedriver", "wdio-chromedriver-service" ] } **Below is my wdio.config.js file** exports.config = { runner: &#39;local&#39;, specs: \[ &#39;./test/specs/**/*.js&#39; ], exclude: \[ // &#39;path/to/excluded/files&#39; ] maxInstances: 1, capabilities: \[{ maxInstances: 1, appWaitDuration: 100000, appWaitActivity: &#39;*&#39; }], logLevel: &#39;info&#39;, bail: 0, baseUrl: &#39;http://localhost&#39;, path: &#39;/wd/hub&#39;, waitforTimeout: 10000, connectionRetryTimeout: 120000, connectionRetryCount: 3, services: &#39;appium&#39;, port: 4723, framework: &#39;mocha&#39;, reporters: &#39;spec&#39;, mochaOpts: { ui: &#39;bdd&#39;, timeout: 60000 }, } **Below is my yml file install and test phases** phases: install: commands: - npm install -g appium - export APPIUM_VERSION=1.17.0 - avm $APPIUM_VERSION - ln -s /usr/local/avm/versions/$APPIUM_VERSION/node_modules/.bin/appium /usr/local/avm/versions/$APPIUM_VERSION/node_modules/appium/bin/appium.js - cd $DEVICEFARM_TEST_PACKAGE_PATH - echo $DEVICEFARM_TEST_PACKAGE_PATH - npm install test: commands: - echo "Navigate to test source code" - cd $DEVICEFARM_TEST_PACKAGE_PATH/node_modules/* - echo "Start Appium Node test for Qbunk Android" - ls - echo $DEVICEFARM_TEST_PACKAGE_PATH - npm install - echo "Installing wdio" - npm install --save webdriverio @wdio/cli - echo "Installing chai" - npm install --save chai - npx wdio $DEVICEFARM_TEST_PACKAGE_PATH/wdio.conf.js --spec $DEVICEFARM_TEST_PACKAGE_PATH/test/specs/*.js Interesting thing is that this <https://aws.amazon.com/blogs/mobile/testing-mobile-apps-across-hundreds-of-real-devices-with-appium-node-js-and-aws-device-farm/> AWS tutorial I followed and write my test with **wd** working fine on AWS Device farm. Kindly let me know if you need more information. Thanks

device farm test platform Appium Node JS uses android build-tools 21.1.1, can I customize this to use 28.0.3?

Hello, We're looking for Test Farm that will be used for mobile apps automation for our company, I have a question about your product: - In our app, we need to scan the QR Code, as a part of the activation process, do AWS Device Farm provide any way to mock camera input? or API to injecting an image as input? Thanks in advance, Paweł Kamieński Edited by: PawelKa on Apr 27, 2020 11:37 PM

hi, I created a new valid test with the and android app. It took almost half an hour and the tests didn&#39;t start https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/5ce87200-3348-43cd-bcc5-98aa249d23ac/runs/2d5c71cc-98b6-450f-ab31-e3abc4bb0319 or https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/5ce87200-3348-43cd-bcc5-98aa249d23ac/runs/cc72a4b1-29d0-499c-ba08-28d87adf3ddc There is no error message, can you have a look? Thank you, Anastasia

Hi folks, last night my tests failed on iOS because "Could not determine iOS SDK version: Command 'xcrun --sdk iphonesimulator --show-sdk-version' timed out after 15000ms" https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/5ce87200-3348-43cd-bcc5-98aa249d23ac/runs/b1f7aa1f-b39e-442e-8985-1a3c82fdfb4b The exact same test jar and IPA were successful 3 times before. Could you have a look? Thank you, Anastasia

Dear Amazon Team, "Google Pixel 3" (Android 10) devices have issues where "Setup Test" fails after ~5 minutes and all our tests are then skipped. We use Android Instrumentation. Please help with fixing these devices otherwise we can't test on Android 10. Failing devices: https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/6b757315-0576-4692-b457-470bc9d9c4c8/jobs/00000/suites/00000/tests/00000 https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/92a7d52c-6948-44fb-a157-ebaaa14b337e/jobs/00000 It was working on 19.10. though so it might be not all devices here are broken: https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/1d94682e-a55d-4faa-98d6-7240e4ef8410/jobs/00000 Might be connected to https://forums.aws.amazon.com/thread.jspa?threadID=311643&tstart=0 Thank you. Regards, Dragan

Dear Amazon Team, Can you please fix the "Samsung Galaxy J7 (2018)"? There is some fullscreen overlay whenever EditText gets focus and it shows this text on white screen: "Uma imagem vale mais que 1000 palavras" Please check: https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/8d328dfd-fcef-4c36-9417-edc9e9ab22af/jobs/00002/suites/00002/tests/00000 https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/8d328dfd-fcef-4c36-9417-edc9e9ab22af/jobs/00002/suites/00002/tests/00000 Thank you. Regards, Dragan

Dear Amazon Team, Can you please disable animations on "Samsung Galaxy S4(Unlocked)" and "Samsung Galaxy S6 (Verizon)"? Our tests are failing significantly on these 2. Is there a reason why these devices have turned on animations since we would expect they are by default disabled on all devices already? Other devices in our device pool have animations disabled as expected. Samsung Galaxy S4(Unlocked): https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/e048c9af-f684-4c84-a19e-08988a3b34c7/jobs/00005/suites/00002/tests/00017 Samsung Galaxy S6 (Verizon): https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/e048c9af-f684-4c84-a19e-08988a3b34c7/jobs/00001/suites/00004/tests/00001 Thank you. Regards, Dragan

Dear Amazon Team, During the last 2 days Samsung Galaxy S8+ (T-Mobile) device always fails. Setup Suite times out after ~1 minute and all tests are skipped. Can you please check & fix this device? https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/edd0b8a2-813e-42cc-a219-564bfa3507a4/jobs/00001/suites/00000/tests/00000 https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/bfcf56f2-f821-4d69-8be9-90de76ec46a9/jobs/00001/suites/00000/tests/00000 Thank you. Regards, Dragan

Our tests have started to break because the app fails to install on iOS ``` ios-deploy-orig[1178:7239] [ !! ] Can't access app path '/tmp/2019829-1117-wn0bdn.a3byb/Payload/Foo Bar.app' : No such file or directory ``` Our app name has a space in it (changed here for privacy), which appears not to be handled by the version of ios-deploy AWS has installed? Can you confirm which version of ios-deploy is installed? NOTE:- We've tried re-running tests with the exact same artefacts (.ipa, test bundle .zip and Test Spec .yml) which previously passed (last week), but now result in the above failure. Tried on multiple devices too. Logs: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/9d1d510c-bf2e-448d-864d-89e7540735f9/runs/80e84376-20a8-49bb-bc5a-92f88d5ddbcf/jobs/00000 Edited by: rhydiant on Sep 30, 2019 4:23 PM Edited by: rhydiant on Sep 30, 2019 5:58 PM

I have a few newbie questions about AWS Device Farm. 1. Probably a really dumb question, but how do I delete a test run? On the list of test runs under the "Automated tests" tab, I can't seem to find an option to delete a run. Even on the results page, there doesn't seem to be an option to delete the run either. Is there a way to delete a specific run without having to delete the entire project or use a third-party tool? 2. I noticed that when a run doesn't have a 100% pass rate, the results page shows which specific tests passed or failed. But when all tests pass, all the tests are grouped together into a single "Tests Suite." How to I expand the suite to show all test cases? 3. The results page only shows the test specification log when all tests pass. But in test runs where there is a failure, the log doesn't appear. How do I enable the test specification log for all runs? Edited by: DannyChiaIntel on Sep 18, 2019 12:29 PM

I want to run tests in a specific version of the Chrome browser, and the device (e.g. Pixel) has a specific version (e.g. 67) preinstalled. (this is an example - the same applies to e.g. the Samsung Internet browser on Samsung devices) Now I want to install a specific newer version (e.g. the 76) to run my tests in, so I specify it as an auxiliary app. During setup, installation fails with error "Package com.android.chrome is already installed as a default system app". https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-2#/projects/66cd20af-0f7e-492b-a5ce-d81174486a49/runs/5bd0c9e9-7b39-40c5-ad34-2ec67289d887/jobs/00000 I expected this to work like it does with "adb install", or even "adb install -r -d" which allows downgrading versions as well. Is it on purpose that you do not support updating system apps? Do you have a recommendation how to work around that?

Hi, I am using Appium _ TestNG _ java to run my tests. After running my test suite of 2 tests, the AWS test report for my test suite shows only one pass or one fail even though i had 2 tests executed. Also the two tests are in one video. How can I get results broken down for each individual test? Thanks, Jonathan

Since android ver 6 and higher, my app asks to allow navigation stuff at the very beggining of it. So I open the app and this allow-popup appears. On my usual appium test everything is fine, I can click on it, close, rerun app and continue. But when AWS runing my app it doesn't click on that popup. So here's 2 runs. First where android 5 and next is higher. Successful run: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/e9705ee5-4015-478f-b9f1-c62f8b73ef49/runs/331a79f6-dabb-4e7d-96ff-483e7b7be706/jobs/00000 Unsuccessful run: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/e9705ee5-4015-478f-b9f1-c62f8b73ef49/runs/12aa1322-646b-41cb-bc90-af6038b7b558

I made some remote access sessions and a particular run is stuck in Pending status. I have already tried to stop it via aws-cli and it does not seems to be stopped. I tried the Samsung Galaxy S9+ (Unlocked), i was able to swipe but not to click. I ended the session and then i'm stuck at: "Your session is ending. Please wait while we process the results." And in the FAQ: "Similarly, for Remote Access sessions, device minutes are measured from the time it takes to prepare a device to your specification to completely removing any apps and data you placed on the device." https://us-west-2.console.aws.amazon.com/devicefarm/home?region=eu-west-3#/projects/87a3e29c-b102-486f-a985-3c4f59700a6b/sessions/5a572b4d-2c51-4976-afed-c7e668225bdb/00000 I hope i will not be charged for a never ending session .

Hi there, I'm new in that thread, and doing my first steps. I'm getting next error when trying to run test (for android): Failed to install com.bla-bla.bla-bla INSTALL_FAILED_NO_MATCHING_ABIS: Failed to extract native libraries, res=-113 Here's the run: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-east-1#/projects/e9705ee5-4015-478f-b9f1-c62f8b73ef49/runs/4e2589ec-d023-4c8e-ae4f-5b1c13b112cd What am I doing wrong? apk file of my app I've got with the help of EZ explorer Edited by: Sunshineauto on Sep 3, 2019 3:49 AM

Hi, I want to add an existing Test Spec file to my automation triggered by AWS SDK (tool written in C#) to run the execution in custom environment. I was partially successful to add a new YAML file but the uploaded file is empty. And I will upload this YAML again and again on each execution - so the project gets flooded with YAML files. I did not found a way to delete it... My scenario looks like this - upload app - upload test ng file - set execution to "custom env" (**missing**) - set already existing YAML file (**missing**) The documentation was not really helpful on that. Can anybody help? Thanks Norman Edited by: NormanHeinrich on Aug 28, 2019 5:29 AM

Dear Amazon team, We cannot start standard Apple Messages app in our XCTest tests on these devices: iPhone XR, iOS 12 iPhone 8 Plus, iOS 12.1 Please whitelist/install Messages app on these devices so we can successfully run our tests. Use case we need for universal links testing is to open Messages app, paste a universal link in a new chat there then tap on the link to go back to the app - this isn't possible for two devices above since there isn't Messages app on them but it does work nicely on other devices. Failing runs: https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/106c8a2d-2f65-4406-a8ef-df95cb86ad1e/jobs/00000/suites/00001/tests/00000 https://us-west-2.console.aws.amazon.com/devicefarm/home?region=us-west-2#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/b1bb6840-56f9-40d9-acc9-1ea5677ffb31/jobs/00003/suites/00006/tests/00002 For failing devices we see in test log that test fails as soon as we try to start Messages app programmatically but we would expect that test execution continues normally. Log contains: t = 247.89s Open com.apple.MobileSMS t = 247.91s Launch com.apple.MobileSMS t = 248.11s Tear Down ... Thank you very much for the support. Kind Regards, Dragan

Hi support, All works very well for Android app but I still have issues about Appium Node.JS for iOS app :( I have the following error: "Unable to start WebDriverAgent session because of xcodebuild failure: "Could not proxy command to remote server. Original error: Error: read ECONNRESET". Make sure you follow the tutorial at https://github.com/appium/appium-xcuitest-driver/blob/master/docs/real-device-config.md. Try to remove the WebDriverAgentRunner application from the device if it is installed and reboot the device." In local, I was it too and I solved that by following that is explained here: https://support.tricentis.com/community/article.do?number=KB0013770 But how can I do that in DeviceFarm? I cannot open WDA xcproject...! Thanks a lot for your helps! Edited by: LyraMobile on Aug 19, 2019 4:27 AM

Good Day, When using the custom environment and run for example 10 test cases, the result will be displayed in one video and in one test Tests Suite. While running the same run under the standard environment the test result and the video will display for each test case 10 videos in this case. Can we separate the run while using the custom environment? Edited by: Malhaj on Jul 25, 2019 1:07 AM

Dear Amazon Team, Samsung Galaxy J5 4G device can't be stopped. It is in "Stopping..." state for long time now. It blocks all our runs so we cannot run any Android test anymore. Please help. Link to the relevant tests on "Stopping..." device: https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/a5cd451c-6593-4e2d-988d-3cfe012bcdd7/runs/7241d1ef-ab58-4cee-8940-251ffb01e1fa/jobs/00003 Thank you. Edited by: marjanovic on Jul 12, 2019 6:05 AM

LG G7 ThinQ https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/5f107c10-66c0-40b9-a928-18d7ca87ae65/jobs/00002 Device is not in English since yesterday. Please fix. Thank you. Edited by: marjanovic on Jun 24, 2019 1:21 AM

Google Pixel 3 XL https://us-west-2.console.aws.amazon.com/devicefarm/home?#/projects/dc2068f7-4b09-417c-9909-b77f559c9641/runs/8139a2d3-c230-4a48-9fbb-72b6950a5cd3/jobs/00003/suites/00004/tests/00001 Please fix. We need keyboard to be in English otherwise our tests fail on this device. Edited by: marjanovic on Jun 12, 2019 7:39 AM