v23.10.1
Bug Fixes
- #2447 Update shard limit for Arm devices to 200 ()
- #2444 Point authentication error message link to correct/relevant docs ()
CI Changes
- #2445 Pass full commit sha through release command ()
v23.10.0
Bug Fixes
- #2438 Replace hub with GitHub CLI 'gh' for GitHub Actions (schnecle)
Features
- #2437 Add retry configuration to Google Cloud Storage client (schnecle)
v23.07.0
Bug Fixes
- #2404 Set correct max test shards when arm devices are configured ()
v23.06.2
Bug Fixes
- #2392 Update out of date dependencies ()
v23.06.1
Bug Fixes
- #2375 Update google-auth-library-oauth2-http version to 1.17.0 ()
v23.06.0
Bug Fixes
- #2373 Update various dependencies ()
- #2369 Remove usage of secrets to fix failing builds ()
- #2360 Rate limiting issue when downloading flank with no version supplied ()
v23.04.0
Bug Fixes
- #2366 Bringing integration tests back to life ()
- #2364 Removed test case time overhead calculation ()
v23.03.2
Bug Fixes
- #2352 Revert "Fixes missing HTTP header X-Goog-User-Project" ()
v23.03.1
Documentation
- #2351 Update developer details
v23.03.0
Bug Fixes
- #2348 Fixes missing HTTP header X-Goog-User-Project
v23.01.0
Bug Fixes
- #2274 Stop loading entire file into memory to fix OOM errors ()
Features
- #2273 Add billable minutes to JSON output report ()
Tests update
- #2331 Update Google cloud project used in tests ()
CI Changes
- #2326 Update scorecard-action version ()
Documentation
- #2325 Fix list formatting in "Authenticate with a service account" section ()
v22.10.0
Refactor
- #2306 Remove references to JFrog and Bintray
- #2299 Remove references to codecov.io
- #2298 Remove references to BugSnag
- #2287 Remove MixPanel analytics
- #2283 Remove Sentry analytics
v22.04.0
Bug Fixes
- #2189 Adds check for .aab format before querying apkDetails (Xlopec)
Features
- #2241 Use Firebase Test Lab matrix details URL for test matrix webLink (tonybaroneee)
v22.03.0
Bug Fixes
- #2225 Revert jackson upgrade
- #2223 Fix BufferUnderflowException when parsing dex file with Espresso 3.4.0 Fixes #2203
- #2207 Add support for robo-directives
Tests update
- #2185 Update app_many_tests to AndroidX libraries
v21.03.0
CI Changes
- #1641 Add missing subcommands (pawelpasterz)
- #1642 Fix release workflow (pawelpasterz)
- #1640 Fix gradle command (pawelpasterz)
- #1618 Publish Flank-scripts to GitHub (piotradamczyk5)
- #1614 Publish flank to maven central (adamfilipow92, piotradamczyk5)
Documentation
- #1627 Explore test-targets-for-shards SDD (Sloox, piotradamczyk5)
- #1592 Fix SDD for GitHub action (piotradamczyk5)
- #1590 GitHub action SDD proposal (piotradamczyk5)
- #1583 Flank-scripts ops package organization (piotradamczyk5)
- #1566 Flank-scripts command overview (piotradamczyk5)
Features
- #1635 Part 2 of IT tests (Sloox, pawelpasterz)
- #1624 Part 1 of multiple OS IT tests (Sloox)
- #1559 Add storing output as json (adamfilipow92, piotradamczyk5)
Bug Fixes
- #1631 Update compare files in IT (Sloox, pawelpasterz)
- #1628 Remove analytics from integration tests (Sloox)
- #1626 Spacing fix for yml file (Sloox)
- #1620 Authenticate in GitHub CLI before release Flank-Scripts (piotradamczyk5)
- #1619 Releasing flank-scripts (piotradamczyk5)
- #1617 Release job (piotradamczyk5)
- #1612 NPE on dumpShards (adamfilipow92)
- #1593 Remove GameloopIT test for IOS (Sloox)
- #1585 Print Flank version only on run command (adamfilipow92)
- #1581 Fixes for Workflow files & IT tests (Sloox, adamfilipow92)
- #1575 Increase heap size (pawelpasterz)
- #1564 Windows issues (Sloox)
- #1560 Fix IT workflow for windows (Sloox)
- #1557 Move shards json to result directory (adamfilipow92, piotradamczyk5)
Refactor
- #1589 Move function to files which use them (adamfilipow92, piotradamczyk5)
- #1584 Reorganize domain layer of flank scripts (adamfilipow92, piotradamczyk5)
- #1574 Refactor CLI of Flank-scripts (piotradamczyk5)
- #1572 Flank scripts features not depends on each other (adamfilipow92)
- #1533 Refactor flank scripts (jan-gogo, adamfilipow92, piotradamczyk5)
v21.01.0
CI Changes
- #1439 Remove sanity checks and change full suite IT OS to macOS (piotradamczyk5)
- #1433 Change IT workflow comment (pawelpasterz)
- #1424 Auto update firebase api client (piotradamczyk5)
- #1412 1411 implement version (pawelpasterz)
- #1409 Trigger integration tests from pr comment (adamfilipow92, piotradamczyk5)
- #1385 Get rid of binaries submodule (jan-gogo, piotradamczyk5)
- #1400 Change gradle command (pawelpasterz)
- #1398 Add create google account config step (pawelpasterz)
- #1353 Implement workflow for full suite IT (cron + manual) (pawelpasterz, piotradamczyk5)
- #1378 Update GH API (pawelpasterz)
Features
- #1436 1422 autodetect new gcloud features (pawelpasterz)
- #1429 Change uploading file output (adamfilipow92, piotradamczyk5)
- #1426 Support for iOS gameloop tests (Sloox, jan-gogo, adamfilipow92)
- #1421 Default project Id - use GOOGLE_APPLICATION_CREDENTIALS first, then GOOGLE_CLOUD_PROJECT (pawelpasterz)
- #1404 Add basic gameloop IT for Android (Sloox, adamfilipow92)
- #1393 Upload matrix ids (Sloox, jan-gogo)
- #1362 Add logic to verify xml results (adamfilipow92, piotradamczyk5)
- #1321 IOS support for testplans (axelzuziak-gogo, jan-gogo)
- #1375 Add new output style to print smaller output (adamfilipow92, piotradamczyk5)
- #1370 Add support for "fail fast" feature on FTL (RainNapper)
Refactor
- #1432 Small changes in flank-scripts module (pawelpasterz)
- #1418 Create module with shared utilities (piotradamczyk5)
- #1417 Update firebase_apis module (pawelpasterz)
Bug Fixes
- #1414 Add attempts to performance test (jan-gogo)
- #1410 Fix integration tests on CI. (adamfilipow92, piotradamczyk5)
- #1413 Fix calculate shards test (pawelpasterz, jan-gogo)
- #1405 CancelCommandRuns should works independent (Sloox, adamfilipow92)
- #1395 Print args when iOS tests are not found (jan-gogo, adamfilipow92)
- #1387 Fix NPE (pawelpasterz, jan-gogo)
- #1372 Fix Flank Scripts version verification tasks (piotradamczyk5)
Documentation
- #1401 Update orchestrator version in docs (adamfilipow92)
- #1380 Added documentation about handling #1374 issue (piotradamczyk5)
Tests update
- #1316 [IT] Add test cases (pawelpasterz, adamfilipow92)
v20.12.0
Bug Fixes
- #1366 Generation of release notes (piotradamczyk5)
- #1357 Fix release Action (Sloox, piotradamczyk5)
- #1333 Publishing Flank snapshot (Sloox, piotradamczyk5)
- #1338 Fix printing supported versions id (adamfilipow92)
- #1329 Uploading performance metrics for multiple matrices (adamfilipow92, piotradamczyk5)
- #1326 Downloading performance Metrics (adamfilipow92, piotradamczyk5)
- #1323 Nested subdirectories in results-dir (piotradamczyk5)
- #1307 Integration tests on windows (pawelpasterz, adamfilipow92)
- #1301 Handling broken token issues (piotradamczyk5)
- #1298 Fix incorrect printed shards number (pawelpasterz)
- #1295 Print formatted message instead of throwing NPE (pawelpasterz)
- #1283 Integration tests on windows (adamfilipow92, piotradamczyk5)
- #1275 Fix getting PR number on pull_request_review (piotradamczyk5)
- #1272 Fix integration test counter and trigger (piotradamczyk5)
- #1219 #842 support for test targets flag in multiple testbundles xctest (axelzuziak-gogo, jan-gogo)
- #1261 Add additional index to matrix when multiple test runs (pawelpasterz, adamfilipow92)
- #1250 Fix running flank workflows on Windows (piotradamczyk5)
- #1247 Flank web documentation link (piotradamczyk5)
- #1243 Disable cache for :test_runner:shadowJar task (pawelpasterz)
- #1233 Fix withClassName filter (pawelpasterz)
- #1221 Always dump shards (pawelpasterz, adamfilipow92)
- #1236 Empty branch name in workflows (jan-gogo)
- #1232 Test_runner:resolveArtifacts task (jan-gogo)
- #1225 Release job missing env ()
- #1188 Revert "feat: Dump shards and upload on every run (#1171)" (pawelpasterz)
- #1177 Fix flank-scripts test (pawelpasterz)
- #1167 Flank action (Sloox)
- #1161 Release notes slack tweaks (Sloox)
- #1164 Dependabot security issues (Sloox)
Refactor
- #1350 Ios test artifacts cleanup #1345 (axelzuziak-gogo, jan-gogo)
- #1339 Simplify beforeRunTests return signature (jan-gogo)
- #1281 Prepare flank codebase for supporting iOS testplans (axelzuziak-gogo, jan-gogo)
- #1246 Rewrite scripts to Kotlin (adamfilipow92, piotradamczyk5)
CI Changes
- #1361 Publishing and downloading flank scripts (piotradamczyk5)
- #1352 Make flank release more automatically (piotradamczyk5)
- #1344 Add directory filter for IT (pawelpasterz)
- #1310 Copy properties from issue to pull request (adamfilipow92, piotradamczyk5)
- #1320 Added pre-commit-hook (Sloox)
- #1286 Update existing bot comment (pawelpasterz)
- #1269 Add build scan for ubuntu workflow (pawelpasterz)
- #1265 Refresh integration tests (piotradamczyk5)
- #1263 GitHub Action optimizations (piotradamczyk5)
- #1253 Use GitHub app token for creating release notes (piotradamczyk5)
- #1248 Update deprecated options ::set-env and ::add-path (piotradamczyk5)
- #1240 Add current date to dependency PR (pawelpasterz)
- #1148 Add scripts for test artifacts management (jan-gogo, piotradamczyk5)
- #1210 Enable RC versions for gradle (dependency update task) (pawelpasterz)
- #1158 Fix test artifacts url (jan-gogo)
- #1152 Added WSL workflow (Sloox, piotradamczyk5)
Documentation
- #1355 Add documentation about code review by step (jan-gogo)
- #1311 Update documentations & Add Headings (Sloox)
- #1222 Add flank website code (jan-gogo)
- #1229 Update documentation (pawelpasterz)
- #1215 Added documentation about missing options (piotradamczyk5)
- #1190 Added documentation about Cucumber support (piotradamczyk5)
- #1181 Investigation incorrect outcome for flaky tests (pawelpasterz, adamfilipow92)
- #1173 Add FAQ section about sync problems (piotradamczyk5)
- #1131 Investigate flank options (jan-gogo, adamfilipow92)
Features
- #1325 IOS GameLoop addition (Sloox, axelzuziak-gogo)
- #1299 Add test targets for shard (Sloox, pawelpasterz)
- #1292 Added Performance Metrics for Android (piotradamczyk5)
- #1289 Add ios app for gameloop (Sloox)
- #1291 Add --test-special-entitlements for iOS (piotradamczyk5)
- #1266 IOS add directories-to-pull (adamfilipow92, piotradamczyk5)
- #1287 Ios scenario numbers (Sloox)
- #1284 Add Type to iOS (piotradamczyk5)
- #1262 Gameloop addition (Sloox)
- #1273 Add Additional Ipas option to iOS (piotradamczyk5)
- #1267 Add Other Files for iOS (adamfilipow92, piotradamczyk5)
- #1255 Obb file addition (Sloox)
- #1256 Update test shard limits (adamfilipow92)
- #1244 Added scenario-numbers (Sloox)
- #1238 Scenario Labels (Sloox)
- #1230 Add type (Sloox)
- #1186 Grant permissions addition (Sloox)
- #1214 Added printing missing options (piotradamczyk5)
- #1175 Android integration tests run on non wsl windows (Sloox, adamfilipow92)
- #1171 Dump shards and upload on every run (adamfilipow92)
- #1163 Integration tests run when pr is approved and not draft (adamfilipow92, piotradamczyk5)
- #1159 Improve error message on matrix failures (pawelpasterz, adamfilipow92)
- #1133 Auto update dependencies (piotradamczyk5)
Tests update
- #1300 Add flank multi test targets example (jan-gogo)
- #1274 Disable iOS test for Windows (piotradamczyk5)
- #1187 Remove verification test for edge cases of incorrect outcome in flaky tests (adamfilipow92)
- #1174 Added cucumber sample app for testing #1118 (piotradamczyk5)
v7.0.0
- #574 Improve test shard error reporting. Update device catalog to use projectId. (bootstraponline)
- #582 Fix iOS exit code when using flaky-test-attempts. Don't print environment-variables to stdout for security. (bootstraponline)
- #584 Poll all test executions instead of only the first per matrix. (bootstraponline)
- #585 Fix bug in smart flank when sharding tests that run in 0 seconds. (bootstraponline)
v6.2.0
- #560 Add
--test-runner-class
support for Android. (jschear)
v6.1.0
- #542 Add
additional-app-test-apks
to include multiple app/test apk pairs in a single run. (bootstraponline)
v5.1.0
- #537 Add
smart-flank-disable-upload
yml option to prevent new results from overriding previous results. (elihart)
v5.0.0
- #506 Add bugsnag reporting to detect Flank crashes. (bootstraponline)
- #507 Improve error message when credentials fail to load, folder doesn't exist, and on bucket creation failure. Properly pass through user credential when checking the storage bucket. (bootstraponline)
- #514 Rename
testShards
to maxTestShards
(miguelslemos)
- #518 Add deprecation warnings when old key names are used.
flank android doctor --fix
will auto fix the YAML file. (bootstraponline)
- #519 Rename
maxTestShards
to max-test-shards
, shardTime
to shard-time
, repeatTests
to repeat-tests
, smartFlankGcsPath
to smart-flank-gcs-path
, disableSharding
to disable-sharding
. Moved project
from gcloud
to flank
(bootstraponline)
- #523 Add
--local-result-dir
to make it easy to find the test result at a fixed path. (bootstraponline)
- #524 Fix iOS test sharding when there's a space in the path (bootstraponline)
- #522 Correctly report test results and exit code when using
flaky-test-attempts
(bootstraponline)
- #530 Save files downloaded with
files-to-download
to device root folder. Fix web links in HTML report. (bootstraponline)
- #533 Update matrix_ids.json on every matrix change. Useful for debugging runs that crashed. (jschear)
v4.2.0
- Fix create Gcs bucket #444
- Add
files-to-download
to Android and iOS. Specify a list of regular expressions to download files from the Google Cloud Storage bucket. #441
- Add
flank auth login
to authorize with a user account instead of a service account. #446
- Add
flaky-test-attempts
support on Android and iOS which automatically retries failed tests. #454
v4.1.1
- Add support for parameterized iOS tests when shard count is 1. #435
- Add support for
SkipTestIdentifiers
when parsing iOS .xctestrun
files. #435
v4.1.0
app
, test
, and xctestrun-file
now support ~
, environment variables, and globs (*
, **
) when resolving paths. #386
- Add CLI support for
flank android run
and flank ios run
- Add experimental
smartFlankGcsPath
to shard iOS and Android tests by time using historical run data. The amount of shards used is set by testShards
. #385
- Fix parsing empty testcase #402
- Add progress bar when uploading files. #403
- iOS iPhone 8 default device has been updated from 11.2 to 12.0.
v4.0.0
- Add
flank cancel
command. See flank cancel --help
- Add
flank refresh
command. See flank refresh --help
- Automatically detect projectId from service account credential
- Always generate aggregated JUnit XML reports
- Update HTML report to use create-react-app v2.1.0
- Add support for Xcode 10.1
- Improve test parsing to detect Swift tests that throw errors
- Update Flank exit codes, see readme for details.
- Link to matrix root when testing on multiple devices