Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't use progress_hook with DataLakeFileClient.upload_data #39026

Open
rohanjain101 opened this issue Jan 3, 2025 · 2 comments
Open

Can't use progress_hook with DataLakeFileClient.upload_data #39026

rohanjain101 opened this issue Jan 3, 2025 · 2 comments
Assignees
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. feature-request This issue requires a new behavior in the product in order be resolved. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)

Comments

@rohanjain101
Copy link

  • Package Name: azure-storage-file-datalake
  • Package Version: 12.18.0
  • Operating System: Windows 11
  • Python Version: 3.11.7

Describe the bug
azure.storage.filedatalake.aio.DataLakeFileClient doesn't support progress_hook parameter, when using it, there is error like:

    def request(
        self, method: str, url: StrOrURL, **kwargs: Any
    ) -> "_RequestContextManager":
        """Perform HTTP request."""
>       return _RequestContextManager(self._request(method, url, **kwargs))
E       TypeError: ClientSession._request() got an unexpected keyword argument 'progress_hook'

But the upload_data_chunks supports progress_hook.

To Reproduce

    from azure.core import MatchConditions

    async def passthrough(total, received):
        print((total, received))
    async with AsyncDataLakeFileClient(
       account_url,
        "filesystem",
        "/local/users/testdata/file.txt",
        creds,
    ) as fc:
        await fc.upload_data(
            b"1,2,3",
            overwrite=True,
            match_condition=MatchConditions.IfMissing,
            max_concurrency=12,
            progress_hook=passthrough,
        )

Expected behavior
progress_hook to work without error.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

@github-actions github-actions bot added customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jan 3, 2025
@xiangyan99 xiangyan99 added Storage Storage Service (Queues, Blobs, Files) Service Attention Workflow: This issue is responsible by Azure service team. Data Lake Storage Gen2 and removed needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. Data Lake Storage Gen2 labels Jan 3, 2025
@github-actions github-actions bot added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Jan 3, 2025
Copy link

github-actions bot commented Jan 3, 2025

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jalauzon-msft @vincenttran-msft.

@weirongw23-msft
Copy link
Member

Hi @rohanjain101 Rohan, it looks like we do not support progress_hook in our Datalake packages yet. We only added support to our blob package so far and _upload_data_chunks is shared code with blob. We will look into adding support for progress_hook in Datalake in the future.

@weirongw23-msft weirongw23-msft added feature-request This issue requires a new behavior in the product in order be resolved. and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
customer-reported Issues that are reported by GitHub users external to the Azure organization. feature-request This issue requires a new behavior in the product in order be resolved. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

5 participants