nvcc --version in Sagemaker Studio Jupyterlab

Hi there,

I am trying to run the Mamba model from Huggingface as well as some other ML models and am running in the error that I cannot install nvcc.

`Collecting flash-attn==1.0.5 (from lost_in_the_middle==0.0.0) Using cached flash_attn-1.0.5.tar.gz (2.0 MB) Installing build dependencies ... done Getting requirements to build wheel ... error error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> [29 lines of output]

  torch.__version__  = 2.3.0+cu121
  
  
  Traceback (most recent call last):
    File "/home/sagemaker-user/.conda/envs/lost-in-the-middle/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/home/sagemaker-user/.conda/envs/lost-in-the-middle/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/sagemaker-user/.conda/envs/lost-in-the-middle/lib/python3.9/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
    File "/tmp/pip-build-env-sh4wszxj/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
    File "/tmp/pip-build-env-sh4wszxj/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
      self.run_setup()
    File "/tmp/pip-build-env-sh4wszxj/overlay/lib/python3.9/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 109, in <module>
    File "<string>", line 24, in get_cuda_bare_metal_version
    File "/home/sagemaker-user/.conda/envs/lost-in-the-middle/lib/python3.9/subprocess.py", line 424, in check_output
      return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
    File "/home/sagemaker-user/.conda/envs/lost-in-the-middle/lib/python3.9/subprocess.py", line 505, in run
      with Popen(*popenargs, **kwargs) as process:
    File "/home/sagemaker-user/.conda/envs/lost-in-the-middle/lib/python3.9/subprocess.py", line 951, in __init__
      self._execute_child(args, executable, preexec_fn, close_fds,
    File "/home/sagemaker-user/.conda/envs/lost-in-the-middle/lib/python3.9/subprocess.py", line 1837, in _execute_child
      raise child_exception_type(errno_num, err_msg, err_filename)
  FileNotFoundError: [Errno 2] No such file or directory: /usr/local/cuda/bin/nvcc
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully. │ exit code: 1 ╰─> See above for output.`

I have Cuda installed, although !nvidia-smi tells me I have version 12.2

PyTorch Version: 2.0.0.post200 CUDA is available: True CUDA Version: 11.2 Tesla T4

Is this mainly a permissions issue or is there something else I can do to get nvcc running in my Jupyterlab notebook?