Analysis step error, not enough memory

Electromagnetics
1

I am using the following steps to setup my Anaconda/python environment to run an FDEM casing model:

  1. “git clone GitHub - simpeg-research/heagy-2021-tle-casing”
  2. cd heagy-2021-tle-casing
  3. set -scipy<=1.6 in environment.yml
  4. conda init
  5. create a new terminal window and go to that for further commands
  6. conda env create -f environment.yml
  7. conda activate heagy-2021-tle-casing
  8. pip uninstall empymod
  9. pip install empymod==2.2.0
  10. pip install --upgrade simpeg==0.21.1

I am running a big mesh for a CSEM source-receiver configuration with steel casing at r=0. The code takes ~ 350GM of memory on a machine with ~ 750 GB of memory. When I increase the casing OD so that the mesh size increases, but still 100’s of GB below the machine max memory, I get the following crash:

Code:
%%time

fields_dict = {}
dpred={}
for key, m in models.items():
if key not in fields_dict.keys():
t = time.time()
fields_dict[key] = sim.fields(m)
dpred[key]=sim.dpred(m)
print(f" … done {key}. {time.time()-t:1.2e} s")

Crash:

PardisoError Traceback (most recent call last)

File :6

File SimPEG/electromagnetics/frequency_domain/simulation.py:178, in BaseFDEMSimulation.fields(self, m)

176 A = self.getA(freq)

177 rhs = self.getRHS(freq)

→ 178 Ainv = self.solver(A, **self.solver_opts)

179 u = Ainv * rhs

180 if not self.forward_only:

21 self.A = A

22 self.set_kwargs(**kwargs)

—> 23 self.solver = MKLPardisoSolver(

24 self.A,

25 matrix_type=self._matrixType(),

26 factor=False

27 )

File pydiso/mkl_solver.pyx:305, in pydiso.mkl_solver.MKLPardisoSolver.init()

File pydiso/mkl_solver.pyx:511, in pydiso.mkl_solver.MKLPardisoSolver._analyze()

PardisoError: Analysis step error, not enough memory

I have tired “conda install mkl=2022.1” but that does not help.

Can anyone suggest how to debug this or if the modules I am using are out of date??
Thanks for any help.