Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rebuild Epoch Parsing #302

Open
superstructor opened this issue May 25, 2021 · 0 comments
Open

Rebuild Epoch Parsing #302

superstructor opened this issue May 25, 2021 · 0 comments

Comments

@superstructor
Copy link
Contributor

The epoch parsing code is in need of significant refactoring and bug fixes. It has led to the following issues:

The relevant code is in ::receive-new-traces, parse-traces, surrounding code in the metamorphic ns and in subs that use the app-db state derived from the epoch parsing.

Step 1: Move ignored-events filtering from event to subs

In ::recieve-new-traces there is a remove to filter ignored events (via settings) before epochs are stored in app-db.

This causes some issues:

  1. Partial epoch states in app-db leading to odd workarounds in subscriptions which are not all handled
  2. If the user removes or adds an epoch filter with existing traces in app-db, 10x is in an invalid state as it does not have a correct 'view' of epochs according to the settings.

The remove filter needs to be deleted from ::recieve-new-traces and added to the appropriate subs instead.

Step 2: Change metamorphic/parse-traces so that dispatch-sync is its own epoch

The delimitation of dispatch-sync epochs is not detected in metamorphic/parse-traces. This needs to be fixed so a dispatch-sync is returned as a distinct epoch.

Step 3: Move all parsing from ::recieve-new-traces event to metamorphic ns

::receive-new-traces contains too much data transformation/parsing related code. If this is moved to metamorphic ns it would be easier to write unit tests to debug/check correct behavior and so forth.

New APIs/functions should be considered as part of this move, as should renaming. The current names are certainly not great.

Tests should be added for all the significant epoch parsing use cases/situations.

At least the following should be moved:

Step 4: Performance

Epoch parsing and associated code is currently the biggest hit on 10x performance.

Once more of the code is moved into functions in metamorphic with unit tests various optimisations could be explored such as storing the internal parse state as transient vs persistent collections, doing less string parsing etc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant