Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests fail on platforms that are slow to get an internet connection #628

Open
filipleple opened this issue Dec 6, 2024 · 1 comment
Open
Labels
bug Something isn't working reliability Focus on testing and improving reliability of framework

Comments

@filipleple
Copy link
Member

filipleple commented Dec 6, 2024

Device

OptiPlex 7010, for example

RTE version

any

OSFV version

any

Affected component(s) or functionality

Any tests that try to perform some network action right after booting into OS

Brief summary

Tests fail on platforms that are slow to get an internet connection, if they try to perform some network action right after booting into OS

How reproducible

99%

How to reproduce

Run a test that tries to perform some network action right after booting into OS, e.g., tpm-support.

Expected behavior

It should test the functionality specified by the test case and fail only if the criteria of said functionality are not met

Actual behavior

It fails, because the platform is rather slow and it hadn't established a network connection yet

Link to screenshots or logs

==============================================================================
Tpm-Support
==============================================================================
TPM001.001 TPM Support (firmware) :: This test aims to verify that... ......
Chip detection failed, attempting cbmem log detection

TPM001.001 TPM Support (firmware) :: This test aims to verify that... | FAIL |
Keyword 'Check Internet Connection On Linux' failed after retrying 5 times. The last error was: 'ping: g
oogle-public-dns-a.google.com: Temporary failure in name resolution' does not contain ', 0% packet loss'
------------------------------------------------------------------------------
TPM001.002 TPM Support (Ubuntu) :: Check whether the TPM is initia... | FAIL |
AttributeError: module 'requests.exceptions' has no attribute 'ProtocolError'
------------------------------------------------------------------------------
TPM001.003 TPM Support (Windows) :: Check whether the TPM is initi... | SKIP |
TPM001.003 not supported
------------------------------------------------------------------------------
TPM002.001 Verify TPM version (firmware) :: This test aims to veri... ......
Chip detection failed, attempting cbmem log detection

TPM002.001 Verify TPM version (firmware) :: This test aims to veri... | FAIL |
Keyword 'Check Internet Connection On Linux' failed after retrying 5 times. The last error was: 'ping: g
oogle-public-dns-a.google.com: Temporary failure in name resolution' does not contain ', 0% packet loss'
------------------------------------------------------------------------------
TPM002.002 Verify TPM version (Ubuntu) :: This test aims to verify... | PASS |
------------------------------------------------------------------------------
TPM002.003 Verify TPM version (Windows) :: This test aims to verif... | SKIP |
TPM002.003 not supported
------------------------------------------------------------------------------
TPM003.001 Check TPM Physical Presence Interface (firmware) :: Thi... | FAIL |
Keyword 'Check Internet Connection On Linux' failed after retrying 5 times. The last error was: 'ping: g
oogle-public-dns-a.google.com: Temporary failure in name resolution' does not contain ', 0% packet loss'
------------------------------------------------------------------------------
TPM003.002 Check TPM Physical Presence Interface (Ubuntu) :: This ... | PASS |
------------------------------------------------------------------------------
TPM003.003 Check TPM Physical Presence Interface (Windows) :: This... | SKIP |
TPM003.003 not supported
------------------------------------------------------------------------------
Since the asset has been checkout manually by you prior running this script, it will NOT be checked in a
utomatically. Please return the device when work is finished.
Tpm-Support                                                           | FAIL |
9 tests, 2 passed, 4 failed, 3 skipped
==============================================================================

Additional context

No response

Solutions you've tried

I haven't tried fixing it, but it probably could be resolved with adding some Wait For Internet Connection keyword after booting into OS, checking ip a | grep 192 with a 5m+ timeout

@filipleple filipleple added bug Something isn't working reliability Focus on testing and improving reliability of framework labels Dec 6, 2024
@SebastianCzapla
Copy link
Contributor

Addressed it in this commit:
29a4c53

It swaps order of keywords in a way that makes more sense; we wait for the moment out DUT is assigned an IP, then we check connection quality with ping.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working reliability Focus on testing and improving reliability of framework
Projects
None yet
Development

No branches or pull requests

2 participants