Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Blank outputs generated on AIX #1104

Open
prajwal-ibm opened this issue Nov 28, 2024 · 4 comments
Open

Blank outputs generated on AIX #1104

prajwal-ibm opened this issue Nov 28, 2024 · 4 comments

Comments

@prajwal-ibm
Copy link

generator.get_next_tokens() always returns zero

model used : phi3-mini-128k-instruct-cpu-int4-rtn-block-32

platform.platform()
'AIX-3-00C001034C00-powerpc-64bit-COFF'
platform.processor()
'powerpc'
platform.system()
'AIX'
platform.version()
'7'

Packages versions :-
onnx 1.16.2
onnxruntime 1.20.0
onnxruntime-genai 0.5.0

Thank you in advance!!
@prajwal-ibm prajwal-ibm changed the title Model generates blank output on AIX Blank outputs generated on AIX Nov 28, 2024
@ranjitshs
Copy link

ranjitshs commented Nov 28, 2024 •
edited
Loading

@kunal-vaishnavi @snnn @tianleiwu
I used Phi-3-mini-4k-instruct FP32 model to see whether onnxruntime-genai can be ported to AIX system.
Also using the main branch on onnxruntime.
Phi-3-mini-4k-instruct FP32 model is generated on MAC system and AIX is used as target system.

Config file wise, genai_config and tokenizer config are in json format , so these required files are getting loaded properly,

Below dump doesn't look good .
Since, AIX is BE system so need further debugging .
Also, what are the information exchanged between genai to onnxruntime .
Do you think any data/information needs bytes-swapping.

 model_output_values   
Name: logits Shape[ 1 9 32064 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... -NaNQ -NaNQ -NaNQ -NaNQ -NaNQ ]
Name: present.0.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0 0 0 0 0 ... 0 0 0 0 0 ]
Name: present.0.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0 0 0 0 0 ... 0 0 0 0 0 ]
Name: present.1.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0 0 0 0 0 ... 0 0 0 0 0 ]
Name: present.1.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0 0 0 0 0 ... 0 0 0 0 0 ]
Name: present.2.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.2.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.3.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.3.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.4.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.4.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.5.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.5.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.6.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.6.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.7.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.7.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.8.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.8.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.9.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.9.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.10.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.10.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.11.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.11.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.12.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.12.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.13.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.13.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.14.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.14.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.15.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.15.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.16.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.16.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.17.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.17.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.18.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.18.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.19.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.19.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.20.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.20.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.21.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.21.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.22.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.22.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.23.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.23.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.24.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.24.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.25.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.25.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.26.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.26.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.27.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.27.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.28.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.28.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.29.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.29.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.30.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.30.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.31.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]
Name: present.31.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ NaNQ NaNQ NaNQ NaNQ NaNQ ... 0 0 0 0 0 ]

@skyline75489
Copy link
Contributor

What ORT version do you use? Does ORT itself have official support for AIX? Most of the heavy-lifting work is handled in ORT. I honestly don't know about the status of AIX support of ORT.

@ranjitshs
Copy link

ranjitshs commented Nov 29, 2024 •
edited
Loading

Did more debugging to see whether external data (model layers weight) is loaded/parsed properly or not.
External data will be of type int or float16 type for FP32 and FP16 model
It seems, ORT is having some issues and external data is not swapped during loading on BE system.
So I converted the data in ORT code GetExtDataFromTensorProto() and It's seems working for both FP32/16 model.

model_output_values   
Name: logits Shape[ 1 11 32064 ] Type: f Location: CPU
Values[ 19.5032 17.4143 20.6179 19.0061 16.051 ... 20.6433 20.6369 20.6383 20.64 20.6405 ]
Name: present.0.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.254236 0.285825 -0.265658 -0.0545148 -0.0384082 ... 0 0 0 0 0 ]
Name: present.0.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00137585 -0.0118618 0.0116067 -0.00335813 -0.00192426 ... 0 0 0 0 0 ]
Name: present.1.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -2.10918 1.03083 -1.98874 -0.749748 1.47431 ... 0 0 0 0 0 ]
Name: present.1.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0201276 0.0196839 0.0112508 0.0336711 -0.00792556 ... 0 0 0 0 0 ]
Name: present.2.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.895102 1.6975 0.775604 3.12376 -1.01428 ... 0 0 0 0 0 ]
Name: present.2.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0886819 -0.0356335 0.0277562 0.0198591 -0.0758996 ... 0 0 0 0 0 ]
Name: present.3.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0690136 -0.0253172 -0.0307189 -0.0311995 -0.00790528 ... 0 0 0 0 0 ]
Name: present.3.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00299299 0.000449531 0.000901573 -0.00083348 0.00241018 ... 0 0 0 0 0 ]
Name: present.4.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 1.02494 0.261593 -0.785569 0.615066 -0.255092 ... 0 0 0 0 0 ]
Name: present.4.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00656431 0.00497555 0.00505479 -0.00332663 0.000271542 ... 0 0 0 0 0 ]
Name: present.5.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00985932 -0.0171346 0.00192433 -0.00222946 0.012802 ... 0 0 0 0 0 ]
Name: present.5.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.000304519 -0.0036626 -0.00218763 0.00363242 0.00470299 ... 0 0 0 0 0 ]
Name: present.6.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00216228 0.00642922 0.00673114 0.00168049 0.00317869 ... 0 0 0 0 0 ]
Name: present.6.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00231217 0.0147819 -0.0285028 -0.0209285 0.0130454 ... 0 0 0 0 0 ]
Name: present.7.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0125484 0.00347344 -0.0299456 0.00548814 -0.013309 ... 0 0 0 0 0 ]
Name: present.7.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0215421 -0.0127142 -0.0146324 0.0281124 -0.000499426 ... 0 0 0 0 0 ]
Name: present.8.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -7.9629e-05 0.0150982 0.0102033 -0.0143266 0.0155461 ... 0 0 0 0 0 ]
Name: present.8.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.000304141 -0.0190725 0.0460841 -0.00840393 8.41446e-05 ... 0 0 0 0 0 ]
Name: present.9.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -3.30806e-05 6.69989e-05 -0.00242201 0.000680573 -0.00360779 ... 0 0 0 0 0 ]
Name: present.9.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0150728 -0.0163258 0.0162739 0.00720605 0.00516546 ... 0 0 0 0 0 ]
Name: present.10.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00558238 -0.00215721 -0.00179339 0.00730722 0.00895234 ... 0 0 0 0 0 ]
Name: present.10.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00809716 -0.0490146 0.0104217 -0.0183469 -0.0236777 ... 0 0 0 0 0 ]
Name: present.11.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0227879 -0.00578365 0.00261178 0.00553118 0.0104084 ... 0 0 0 0 0 ]
Name: present.11.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0086582 -0.00319619 -0.00839286 0.00774312 0.00382874 ... 0 0 0 0 0 ]
Name: present.12.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0023018 -0.000115494 -0.000992161 0.0105492 0.00458057 ... 0 0 0 0 0 ]
Name: present.12.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0223382 -0.00292525 0.0126353 -0.00229196 -0.0211711 ... 0 0 0 0 0 ]
Name: present.13.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00831568 0.000302838 0.000835411 -0.00234183 -0.0145248 ... 0 0 0 0 0 ]
Name: present.13.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00587512 0.0151342 -0.0129383 0.0058841 -0.0106707 ... 0 0 0 0 0 ]
Name: present.14.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00166867 0.00447156 -0.0082678 0.00300909 -0.00117675 ... 0 0 0 0 0 ]
Name: present.14.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.107471 0.15215 -0.243358 -0.0602169 0.109513 ... 0 0 0 0 0 ]
Name: present.15.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00140902 0.0110362 -0.00757615 -0.0101297 0.00627547 ... 0 0 0 0 0 ]
Name: present.15.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0187795 0.0290469 0.0526959 -0.00667681 0.0256415 ... 0 0 0 0 0 ]
Name: present.16.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0127814 0.00110486 -0.0262131 -0.00921035 -0.000609075 ... 0 0 0 0 0 ]
Name: present.16.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00449384 0.00809387 -0.0240867 -0.00130485 -3.56587e-05 ... 0 0 0 0 0 ]
Name: present.17.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0195802 -0.00431788 -0.000502552 -0.000450039 -0.00359074 ... 0 0 0 0 0 ]
Name: present.17.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.215532 -0.00568823 -0.0113498 -0.0620717 0.0182427 ... 0 0 0 0 0 ]
Name: present.18.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00233065 -0.0211819 0.00780215 0.00751172 -0.0109008 ... 0 0 0 0 0 ]
Name: present.18.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0219227 -0.0385069 -0.0616627 -0.0562806 -0.0477268 ... 0 0 0 0 0 ]
Name: present.19.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0174504 -0.000309858 0.00267176 0.0144675 -0.0050488 ... 0 0 0 0 0 ]
Name: present.19.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00779939 0.00670814 -0.0122448 0.00906955 -0.0508001 ... 0 0 0 0 0 ]
Name: present.20.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0162605 -0.00112878 -0.00964545 -0.000934093 0.016838 ... 0 0 0 0 0 ]
Name: present.20.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0168249 0.00889024 -0.0301857 0.0197288 -0.0188361 ... 0 0 0 0 0 ]
Name: present.21.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00264816 -0.00283724 -0.0100102 -0.00824387 -0.00221259 ... 0 0 0 0 0 ]
Name: present.21.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00884004 0.16288 0.0148322 -0.103825 0.0629525 ... 0 0 0 0 0 ]
Name: present.22.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00192377 -0.00813456 -0.0103524 -0.0201171 -0.0125708 ... 0 0 0 0 0 ]
Name: present.22.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0161572 -0.00455613 -0.0226239 -0.00309558 0.0893283 ... 0 0 0 0 0 ]
Name: present.23.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.022027 0.0018516 0.0101509 0.00825459 0.0137429 ... 0 0 0 0 0 ]
Name: present.23.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00734171 0.0198167 -0.0066022 0.0424827 0.0407197 ... 0 0 0 0 0 ]
Name: present.24.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0147097 0.0150944 0.000950847 0.00515965 -0.000685959 ... 0 0 0 0 0 ]
Name: present.24.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0708422 -0.0724135 -0.0358457 0.0236604 -0.0479103 ... 0 0 0 0 0 ]
Name: present.25.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00787787 -0.0162551 0.00460484 -0.00531378 -0.00634484 ... 0 0 0 0 0 ]
Name: present.25.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00766606 0.00586146 0.0759845 -0.0102314 -0.00870442 ... 0 0 0 0 0 ]
Name: present.26.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.00596797 -0.00416121 0.00173569 -0.0148073 0.00352936 ... 0 0 0 0 0 ]
Name: present.26.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0196702 0.0106482 -0.0559098 -0.0884595 -0.0267909 ... 0 0 0 0 0 ]
Name: present.27.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0145797 -0.00895968 0.00237181 0.0227294 0.0125896 ... 0 0 0 0 0 ]
Name: present.27.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.0278106 -0.000361919 -0.0248885 0.0405614 -0.0238001 ... 0 0 0 0 0 ]
Name: present.28.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0208624 -0.0127186 -0.00104423 0.00720733 -0.00456734 ... 0 0 0 0 0 ]
Name: present.28.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00285507 -0.0251122 -0.0287729 -0.026214 -0.0213298 ... 0 0 0 0 0 ]
Name: present.29.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0105539 0.0272181 -0.0254072 0.018309 0.00858812 ... 0 0 0 0 0 ]
Name: present.29.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.163886 -0.0797483 0.00565815 0.0184726 -0.00365725 ... 0 0 0 0 0 ]
Name: present.30.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0547117 0.022356 0.0317146 -0.00197276 0.00829611 ... 0 0 0 0 0 ]
Name: present.30.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ -0.00977828 0.37553 0.0272748 0.0110753 0.0930018 ... 0 0 0 0 0 ]
Name: present.31.key Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0232148 0.0698595 -0.0103118 -0.0417754 0.0329229 ... 0 0 0 0 0 ]
Name: present.31.value Shape[ 1 32 2048 96 ] Type: f Location: CPU
Values[ 0.0866278 -0.0638338 0.243978 1.40031 -0.0529445 ... 0 0 0 0 0 ]

@snnn @tianleiwu @kunal-vaishnavi @prajwal-ibm
I will track external data load issue in onnxruntime repo separately.

@ranjitshs
Copy link

ranjitshs commented Nov 29, 2024 •
edited
Loading

What ORT version do you use? Does ORT itself have official support for AIX? Most of the heavy-lifting work is handled in ORT. I honestly don't know about the status of AIX support of ORT.

Yes. In AIX, only inference support is available for ORT.
Current support is to train/generate the model in LE system and do the inference in AIX.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@skyline75489 @ranjitshs @prajwal-ibm