-
Notifications
You must be signed in to change notification settings - Fork 382
Open
Labels
bugSomething isn't workingSomething isn't working
Description
Bug Description
Trying to export the model using torch.export.export()..
Trying torch.export._trace._export to trace the graph since torch.export.export() failed
WARNING:py.warnings:/opt/conda/envs/py310/lib/python3.10/copyreg.py:101: FutureWarning: `isinstance(treespec, LeafSpec)` is deprecated, use `isinstance(treespec, TreeSpec) and treespec.is_leaf()` instead.
return cls.__new__(cls, *args)
WARNING:py.warnings:/workspace/TensorRT/tools/llm/cache_utils.py:126: FutureWarning: `treespec.children_specs` is deprecated. Use `treespec.child(index)` to access a single child, or `treespec.children()` to get all children.
in_spec_for_args = in_spec.children_specs[0]
WARNING:py.warnings:/workspace/TensorRT/tools/llm/cache_utils.py:134: FutureWarning: `treespec.children_specs` is deprecated. Use `treespec.child(index)` to access a single child, or `treespec.children()` to get all children.
in_spec_for_args.children_specs.append(_LEAF_SPEC)
WARNING:py.warnings:/workspace/TensorRT/tools/llm/cache_utils.py:139: FutureWarning: `treespec.children_specs` is deprecated. Use `treespec.child(index)` to access a single child, or `treespec.children()` to get all children.
for child_spec in spec.children_specs:
Traceback (most recent call last):
File "/workspace/TensorRT/tools/llm/run_llm.py", line 323, in <module>
trt_model = compile_torchtrt(model, input_ids, args)
File "/workspace/TensorRT/tools/llm/run_llm.py", line 134, in compile_torchtrt
trt_model = torch_tensorrt.dynamo.compile(
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch_tensorrt/dynamo/_compiler.py", line 764, in compile
gm = post_lowering(gm, settings)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch_tensorrt/dynamo/lowering/passes/_aten_lowering_pass.py", line 139, in post_lowering
fake_tensor_updater.incremental_update(fake_mode)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch_tensorrt/dynamo/lowering/passes/_FakeTensorUpdater.py", line 193, in incremental_update
new_fake_tensor = node.target(*args, **kwargs)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_ops.py", line 819, in __call__
return self._op(*args, **kwargs)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/utils/_stats.py", line 29, in wrapper
return fn(*args, **kwargs)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1397, in __torch_dispatch__
return self.dispatch(func, types, args, kwargs)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 2155, in dispatch
return self._cached_dispatch_impl(func, types, args, kwargs)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1548, in _cached_dispatch_impl
entry = self._make_cache_entry(state, key, func, args, kwargs, output)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1969, in _make_cache_entry
output_info = self._get_output_info_for_cache_entry(
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 1863, in _get_output_info_for_cache_entry
synth_output = self._output_from_cache_entry(
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 2076, in _output_from_cache_entry
return self._get_output_tensor_from_cache_entry(
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/_subclasses/fake_tensor.py", line 2031, in _get_output_tensor_from_cache_entry
empty = torch.empty_strided(
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/fx/experimental/sym_node.py", line 562, in expect_true
return self.shape_env.guard_or_defer_runtime_assert(
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/fx/experimental/recording.py", line 273, in wrapper
return retlog(fn(*args, **kwargs))
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 7779, in guard_or_defer_runtime_assert
static_expr = self._maybe_evaluate_static(expr)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 2636, in wrapper
return fn_cache(self, *args, **kwargs)
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 6437, in _maybe_evaluate_static
r = _maybe_evaluate_static_worker(
File "/opt/conda/envs/py310/lib/python3.10/site-packages/torch/fx/experimental/symbolic_shapes.py", line 2421, in _maybe_evaluate_static_worker
assert vr is not None
AssertionError
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Environment
Build information about Torch-TensorRT can be found by turning on debug messages
- Torch-TensorRT Version (e.g. 1.0.0):
- PyTorch Version (e.g. 1.0):
- CPU Architecture:
- OS (e.g., Linux):
- How you installed PyTorch (
conda,pip,libtorch, source): - Build command you used (if compiling from source):
- Are you using local sources or building from archives:
- Python version:
- CUDA version:
- GPU models and configuration:
- Any other relevant information:
Additional context
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working