[tvos][arm64] Segfault in System.Formats.Tests

[matouskozak]

tvos_arm64_release_allsubsets_mono job is crashing during execution of System.Formats.Tar.Tests

First failing rolling-build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=657938
(commit e1928e4)

Stack trace:

[06:05:15.3374550] 2024-04-27 03:05:15.386 System.Formats.Tar.Tests[12317:83785660] 	[PASS] System.Formats.Tar.Tests.UstarTarEntry_Conversion_Tests.Constructor_ConversionPax_BackAndForth_CharacterDevice
[06:05:15.3432840] 
[06:05:15.3433050] =================================================================
[06:05:15.3433130] 	Native Crash Reporting
[06:05:15.3433190] =================================================================
[06:05:15.3433250] Got a SIGSEGV while executing native code. This usually indicates
[06:05:15.3433310] a fatal error in the mono runtime or one of the native libraries 
[06:05:15.3433400] used by your application.
[06:05:15.3433460] =================================================================
[06:05:15.3468570] Xamarin.Hosting: Process '12317' exited with exit code  or crashing signal 6.

Known Issue Error Message

Fill the error message using step by step known issues guidance.

{
  "ErrorMessage": "",
  "ErrorPattern": "",
  "BuildRetry": false,
  "ExcludeConsoleLog": false
}

Report

Summary

24-Hour Hit Count	7-Day Hit Count	1-Month Count
0	0	0

[dotnet-policy-service]

Tagging subscribers to 'os-tvos': @steveisok, @akoeplinger
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-formats-tar
See info in area-owners.md if you want to be subscribed.

[carlossanlop]

There have been no changed in Tar recently, so if this is the commit that started it, we should change the label:

First failing rolling-build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=657938
(commit e1928e4)

[dotnet-policy-service]

Tagging subscribers to this area: @directhex
See info in area-owners.md if you want to be subscribed.

[matouskozak]

There have been no changed in Tar recently, so if this is the commit that started it, we should change the label:

First failing rolling-build: https://dev.azure.com/dnceng-public/public/_build/results?buildId=657938
(commit e1928e4)

I agree. To clarify, the linked build / commit is the first rolling-build I've found to fail, it isn't necessary the one causing the failure.

[matouskozak]

Another crash:

[19:01:22.0110020] 2024-08-14 16:01:22.136 System.Formats.Tar.Tests[77003:107144384] 	[PASS] System.Formats.Tar.Tests.TarReader_GetNextEntry_Tests.MalformedArchive_HeaderSize
[19:01:22.0228220] 
[19:01:22.0228660] =================================================================
[19:01:22.0228770] 	Native Crash Reporting
[19:01:22.0228850] =================================================================
[19:01:22.0229200] Got a SIGBUS while executing native code. This usually indicates
[19:01:22.0229340] a fatal error in the mono runtime or one of the native libraries 
[19:01:22.0229410] used by your application.
[19:01:22.0229480] =================================================================

I wasn't able to reproduce this locally on iOS device, possibly tvOS only?

[kotlarmilos]

I managed to reproduce it on iOS. Here are more details:

(lldb) bt
* thread #12, name = '.NET TP Worker', stop reason = signal SIGBUS
  * frame #0: 0x00000002233aef20 libsystem_platform.dylib`__bzero + 64

(lldb) register read
General Purpose Registers:
        x0 = 0x000000016c3e5950
        x1 = 0x0000000000000000
        x2 = 0x00000000ffff1b0f
        x3 = 0x000000016c3f4000
        x4 = 0x000000016c3e59c0
        x5 = 0x0000000000000008
        x6 = 0x0000000000000000
        x7 = 0x0000000106402f38  System.Formats.Tar.Tests`builtin_types
        x8 = 0x00000001000001fe
        x9 = 0x00000001000001ff
       x10 = 0x0000000000000001
       x11 = 0x0000000000000000
       x12 = 0x0000000000000000
       x13 = 0x0000000000000000
       x14 = 0x00000000000000e6
       x15 = 0x00000001110138b0
       x16 = 0x00000002233aeee0  libsystem_platform.dylib`__bzero
       x17 = 0x0000000000000000
       x18 = 0x0000000000000000
       x19 = 0x000000010e0f1f70
       x20 = 0x000000016c3e5950
       x21 = 0x00000000000001ff
       x22 = 0x0000000000000200
       x23 = 0x0000000000000001
       x24 = 0x000000030017d860
       x25 = 0x000000010e0f2088
       x26 = 0x0000000000000000
       x27 = 0x000000010b0a2800
       x28 = 0x0000000106402f38  System.Formats.Tar.Tests`builtin_types
        fp = 0x000000016c3e5b70
        lr = 0x0000000104e5d07c  System.Formats.Tar.Tests`System_Formats_Tar_System_Formats_Tar_TarHeader_WriteEmptyPadding_System_IO_Stream + 92
        sp = 0x000000016c3e5950
        pc = 0x00000002233aef20  libsystem_platform.dylib`__bzero + 64
      cpsr = 0x20001000

The failure occurs in a thread, which suggests it might be caused in an async call. Additionally, it may be related to WriteEmptyPadding according to the lr reg.

[matouskozak]

Great investigation @kotlarmilos. Does it reproduce just by running the test on iOS device or is there a special setup needed? I wonder why I couldn't reproduce it before.

@BrzVlad Do you have any pointers on what could be the cause of failure based on Milos' report?