UI/BE: Send to Data Curation

[rosibaj]

• Add UI flow for users to send to curation
• Once submitter sends to curation, add to a curation bucket. the TSV/FASTA files can be named based on study_id/egoid to identify the attempted submission
• When objects are added to the bucket, notify the data curation team.

OPTION 1: use collab, deploy KAFKA write to KAFKA queue, and read send email to helpdesk.
OPTION 2: use aws, pipeline and sns publish. free tier possible can work.

Change the data ingestion process

Currently: metadata/fasta uploaded → validated → reformatted → submitted to Song/Score → indexed by Maestro → released to Portal
Modified to: metadata/fasta uploaded → validated → stored in S3 bucket → curator reviews within 24 hours
if no problem → curator approves → data reformatted → submitted to Song/Score → indexed → Portal.
Problem case 1 → curator emails submitter and recommends changes → submitter resubmits
Problem case 2 → curator changes data directly → curator approves

[lepsalex]

had a good discussion with Jared re: using OpenStack services to accomplish this task, he will look into feasibility and review with me, will move on to other tickets for now.

[b-f-chan]

My opinion is this is a large feature and requires some re-write of the submission process as well as a change in the business process itself. I don't think this is in the scope of the 3 new work packages. We'll confirm with @scottcain however, but this is our strong recommendation