Installation with pip results in broken installation
### Description

This is reopening #7253 since it still persists with CUDA 12.1 and cupy-cuda12x on Python 3.11 in a fresh environment. CC @kmaehashi 

Importing `cupy` fails with an `ImportError`:

```
In [1]: import cupy
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
File ~\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\__init__.py:17
     16 try:
---> 17     from cupy import _core  # NOQA
     18 except ImportError as exc:

File ~\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\_core\\__init__.py:3
      1 # mypy: ignore-errors
----> 3 from cupy._core import core  # NOQA
      4 from cupy._core import fusion  # NOQA

File cupy\\_core\\core.pyx:1, in init cupy._core.core()

File ~\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\cuda\\__init__.py:8
      7 from cupy._environment import get_hipcc_path  # NOQA
----> 8 from cupy.cuda import compiler  # NOQA
      9 from cupy.cuda import device  # NOQA

File ~\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\cuda\\compiler.py:13
     11 import warnings
---> 13 from cupy.cuda import device
     14 from cupy.cuda import function

File cupy\\cuda\\device.pyx:1, in init cupy.cuda.device()

ImportError: DLL load failed while importing runtime: The specified module could not be found.

The above exception was the direct cause of the following exception:

ImportError                               Traceback (most recent call last)
Cell In[1], line 1
----> 1 import cupy

File ~\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\__init__.py:19
     17     from cupy import _core  # NOQA
     18 except ImportError as exc:
---> 19     raise ImportError(f'''
     20 ================================================================
     21 {_environment._diagnose_import_error()}
     22
     23 Original error:
     24   {type(exc).__name__}: {exc}
     25 ================================================================
     26 ''') from exc
     29 from cupy import cuda  # NOQA
     30 # Do not make `cupy.cupyx` available because it is confusing.

ImportError:
================================================================
Failed to import CuPy.

If you installed CuPy via wheels (cupy-cudaXXX or cupy-rocm-X-X), make sure that the package matches with the version of CUDA or ROCm installed.

On Linux, you may need to set LD_LIBRARY_PATH environment variable depending on how you installed CUDA/ROCm.
On Windows, try setting CUDA_PATH environment variable.

Check the Installation Guide for details:
  https://docs.cupy.dev/en/latest/install.html

CUDA Path: C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.1
DLL dependencies:
  KERNEL32.dll -> C:\\WINDOWS\\System32\\KERNEL32.DLL
  MSVCP140.dll -> C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\MSVCP140.dll
  VCRUNTIME140.dll -> C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\VCRUNTIME140.dll
  api-ms-win-crt-convert-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-environment-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-filesystem-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-heap-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-runtime-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-stdio-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  cuTENSOR.dll -> not found
  cublas64_12.dll -> not found
  cudart64_12.dll -> not found
  cudnn64_8.dll -> not found
  cufft64_11.dll -> not found
  curand64_10.dll -> not found
  cusolver64_11.dll -> not found
  cusparse64_12.dll -> not found
  nvcuda.dll -> not found
  nvrtc64_120_0.dll -> not found
  python311.dll -> C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\python311.dll

Original error:
  ImportError: DLL load failed while importing runtime: The specified module could not be found.
================================================================
```

The environment variable is set correctly: `CUDA_PATH=C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.1`

### To Reproduce

```py
import cupy
```


### Installation

Wheel (`pip install cupy-***`)

### Environment

```
(spaconv311) C:\\Users\\weber>python -c "import cupy; cupy.show_config()"
Traceback (most recent call last):
  File "C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\__init__.py", line 17, in <module>
    from cupy import _core  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^
  File "C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\_core\\__init__.py", line 3, in <module>
    from cupy._core import core  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "cupy\\_core\\core.pyx", line 1, in init cupy._core.core
  File "C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\cuda\\__init__.py", line 8, in <module>
    from cupy.cuda import compiler  # NOQA
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\cuda\\compiler.py", line 13, in <module>
    from cupy.cuda import device
  File "cupy\\cuda\\device.pyx", line 1, in init cupy.cuda.device
ImportError: DLL load failed while importing runtime: The specified module could not be found.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\Lib\\site-packages\\cupy\\__init__.py", line 19, in <module>
    raise ImportError(f'''
ImportError:
================================================================
Failed to import CuPy.

If you installed CuPy via wheels (cupy-cudaXXX or cupy-rocm-X-X), make sure that the package matches with the version of CUDA or ROCm installed.

On Linux, you may need to set LD_LIBRARY_PATH environment variable depending on how you installed CUDA/ROCm.
On Windows, try setting CUDA_PATH environment variable.

Check the Installation Guide for details:
  https://docs.cupy.dev/en/latest/install.html

CUDA Path: C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v12.1
DLL dependencies:
  KERNEL32.dll -> C:\\WINDOWS\\System32\\KERNEL32.DLL
  MSVCP140.dll -> C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\MSVCP140.dll
  VCRUNTIME140.dll -> C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\VCRUNTIME140.dll
  api-ms-win-crt-convert-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-environment-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-filesystem-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-heap-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-runtime-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  api-ms-win-crt-stdio-l1-1-0.dll -> C:\\WINDOWS\\System32\\ucrtbase.dll
  cuTENSOR.dll -> not found
  cublas64_12.dll -> not found
  cudart64_12.dll -> not found
  cudnn64_8.dll -> not found
  cufft64_11.dll -> not found
  curand64_10.dll -> not found
  cusolver64_11.dll -> not found
  cusparse64_12.dll -> not found
  nvcuda.dll -> not found
  nvrtc64_120_0.dll -> not found
  python311.dll -> C:\\Users\\weber\\miniconda3\\envs\\spaconv311\\python311.dll

Original error:
  ImportError: DLL load failed while importing runtime: The specified module could not be found.
================================================================
```

### Additional Information

_No response_
diff --git a/cupy/_environment.py b/cupy/_environment.py
index fc646b667..397cdff49 100644
--- a/cupy/_environment.py
+++ b/cupy/_environment.py
@@ -200,8 +200,10 @@ def _setup_win32_dll_directory():
     # Setup DLL directory to load CUDA Toolkit libs and shared libraries
     # added during the build process.
     if sys.platform.startswith('win32'):
-        is_conda = ((os.environ.get('CONDA_PREFIX') is not None)
-                    or (os.environ.get('CONDA_BUILD_STATE') is not None))
+        # see _can_attempt_preload()
+        config = get_preload_config()
+        is_conda = (config is not None and (config['packaging'] == 'conda'))
+
         # Path to the CUDA Toolkit binaries
         cuda_path = get_cuda_path()
         if cuda_path is not None:
['cupy/_environment.py']