[WIP] Phi3poc #2301
[WIP] Phi3poc #2301
Conversation
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #2301 +/- ##
=======================================
Coverage 84.48% 84.48%
=======================================
Files 328 328
Lines 16799 16799
Branches 1498 1498
=======================================
+ Hits 14192 14193 +1
+ Misses 2607 2606 -1 ☔ View full report in Codecov by Sentry. |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| import shutil | ||
| import sys | ||
|
|
||
| class Peekable: |
There was a problem hiding this comment.
nit: _PeekableIterator
| return self._cache[:n] | ||
|
|
||
|
|
||
| class ModelParam: |
| return self.param | ||
|
|
||
|
|
||
| class ModelConfig: |
| self.config.update(kwargs) | ||
|
|
||
|
|
||
| def camel_to_snake(text): |
There was a problem hiding this comment.
there might already be one in library to use
| "output column", | ||
| typeConverter=TypeConverters.toString, | ||
| ) | ||
| modelParam = Param(Params._dummy(), "modelParam", "Model Parameters") |
There was a problem hiding this comment.
Maybe explain difference between model params and other params (you can just link to other docs if easier)
| typeConverter=TypeConverters.toString, | ||
| ) | ||
| modelParam = Param(Params._dummy(), "modelParam", "Model Parameters") | ||
| modelConfig = Param(Params._dummy(), "modelConfig", "Model configuration") |
There was a problem hiding this comment.
Maybe explain difference between model config and other params (you can just link to other docs if easier)
| useFabricLakehouse = Param( | ||
| Params._dummy(), | ||
| "useFabricLakehouse", | ||
| "Use FabricLakehouse", |
There was a problem hiding this comment.
If this is for a local cache then you might be able to make the verbage generic like useLocalCache
| deviceMap = Param( | ||
| Params._dummy(), | ||
| "deviceMap", | ||
| "device map", |
There was a problem hiding this comment.
might need to explain a bit more about this param and what it takes
| torchDtype = Param( | ||
| Params._dummy(), | ||
| "torchDtype", | ||
| "torch dtype", |
| def load_model(self): | ||
| """ | ||
| Loads model and tokenizer either from Fabric Lakehouse or the HuggingFace Hub, | ||
| depending on the 'useFabricLakehouse' param. |
There was a problem hiding this comment.
if you name it more generically place that name here
| "Use FabricLakehouse", | ||
| typeConverter=TypeConverters.toBoolean, | ||
| ) | ||
| lakehousePath = Param( |
There was a problem hiding this comment.
might be able to get rid of earlier param just check if this is None
| if self.getUseFabricLakehouse(): | ||
| local_path = ( | ||
| self.getLakehousePath() or f"/lakehouse/default/Files/{model_name}" | ||
| ) |
There was a problem hiding this comment.
switch to just use cachePath and then in our docs well say this is a good place to store things
|
|
||
| if self.getUseFabricLakehouse(): | ||
| local_path = ( | ||
| self.getLakehousePath() or f"/lakehouse/default/Files/{model_name}" |
| def __init__( | ||
| self, | ||
| base_cache_dir="./cache", | ||
| base_url="https://mmlspark.blob.core.windows.net/huggingface/", |
There was a problem hiding this comment.
lets use
%sh
azcopy cp https://mmlspark.blob.core.windows.net/huggingface/blah /lakehouse/blah
There was a problem hiding this comment.
Youy can also put in the mardown cell a little explanation of this and how its just for a speedup otherwise it will download from the huggingface hub
|
|
||
| def _predict_single_chat(self, prompt, model, tokenizer): | ||
| param = self.getModelParam().get_param() | ||
| chat = [{"role": "user", "content": prompt}] |
There was a problem hiding this comment.
if the prompt is a list, then assume its of structure of "chat"
|
|
||
| def _predict_single_chat(self, prompt, model, tokenizer): | ||
| param = self.getModelParam().get_param() | ||
| chat = [{"role": "user", "content": prompt}] |
There was a problem hiding this comment.
| chat = [{"role": "user", "content": prompt}] | |
| if isinstance(prompt, list): | |
| chat = prompt | |
| else: | |
| chat = [{"role": "user", "content": prompt}] | |
Related Issues/PRs
#xxx
What changes are proposed in this pull request?
Briefly describe the changes included in this Pull Request.
How is this patch tested?
Does this PR change any dependencies?
Does this PR add a new feature? If so, have you added samples on website?
website/docs/documentationfolder.Make sure you choose the correct class
estimators/transformersand namespace.DocTablepoints to correct API link.yarn run startto make sure the website renders correctly.<!--pytest-codeblocks:cont-->before each python code blocks to enable auto-tests for python samples.WebsiteSamplesTestsjob pass in the pipeline.