Error using pm.Data as a parameter to distribution

[rpgoldman]

Description of your problem

I have made a pm.Data object out of data from a table. When I use this object as the value of mu in a pm.Normal, I get an error as follows:

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

I am working to extract a minimal example, but meanwhile the error is here:

> /Users/rpg/src/pymc3/pymc3/distributions/distribution.py(82)get_test_val()
     80         if val is None:
     81             for v in defaults:
---> 82                 if hasattr(self, v) and np.all(np.isfinite(self.getattr_value(v))):
     83                     return self.getattr_value(v)
     84         else:

and occurs when trying to get the value of the median attribute from this variable:

pm.Normal('soc_dem', mu=mus, sd=stds, shape=dataset.shape[0])

The value of self.getattr_value(v) is the result of the pm.Data form, which is of the type theano.tensor.sharedvar.TensorSharedVariable.

I don't know whether the issue is that the TensorSharedVariable does not support np.isfinite, or that the code in distribution.py does not try using eval() here.

Using eval() seems to work. In the debugger I see:

ipdb> p np.all(np.isfinite(self.getattr_value(v).eval()))
True

Please provide a minimal, self-contained, and reproducible example.

[Your code here]

Please provide the full traceback.

TypeError                                 Traceback (most recent call last)
<ipython-input-179-4dcff76e0f63> in <module>
      6     mus = pm.Data('mus', np.where(oecd_member, (oecd_gini_mean - gini_mean) / gini_std, (non_oecd_gini_mean - gini_mean) / gini_std))
      7     stds = pm.Data('sigmas', np.where(oecd_member, oecd_gini_std / gini_std, non_oecd_gini_std / gini_std))
----> 8     soc_dem = pm.Normal('soc_dem', mu=mus, sd=stds, shape=dataset.shape[0])
      9 
     10     #soc_dem_gc_factor = pm.HalfCauchy('beta(SD_Gini)', beta=1.0)

~/src/pymc3/pymc3/distributions/distribution.py in __new__(cls, name, *args, **kwargs)
     50             total_size = kwargs.pop('total_size', None)
     51             dist = cls.dist(*args, **kwargs)
---> 52             return model.Var(name, dist, data, total_size)
     53         else:
     54             raise TypeError("Name needs to be a string but got: {}".format(name))

~/src/pymc3/pymc3/model.py in Var(self, name, dist, data, total_size)
    834                 with self:
    835                     var = FreeRV(name=name, distribution=dist,
--> 836                                  total_size=total_size, model=self)
    837                 self.free_RVs.append(var)
    838             else:

~/src/pymc3/pymc3/model.py in __init__(self, type, owner, index, name, distribution, total_size, model)
   1286             self.distribution = distribution
   1287             self.tag.test_value = np.ones(
-> 1288                 distribution.shape, distribution.dtype) * distribution.default()
   1289             self.logp_elemwiset = distribution.logp(self)
   1290             # The logp might need scaling in minibatches.

~/src/pymc3/pymc3/distributions/distribution.py in default(self)
     75 
     76     def default(self):
---> 77         return np.asarray(self.get_test_val(self.testval, self.defaults), self.dtype)
     78 
     79     def get_test_val(self, val, defaults):

~/src/pymc3/pymc3/distributions/distribution.py in get_test_val(self, val, defaults)
     80         if val is None:
     81             for v in defaults:
---> 82                 if hasattr(self, v) and np.all(np.isfinite(self.getattr_value(v))):
     83                     return self.getattr_value(v)
     84         else:

TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

> /Users/rpg/src/pymc3/pymc3/distributions/distribution.py(82)get_test_val()
     80         if val is None:
     81             for v in defaults:
---> 82                 if hasattr(self, v) and np.all(np.isfinite(self.getattr_value(v))):
     83                     return self.getattr_value(v)
     84         else:```



## Versions and main components

* PyMC3 Version: master
* Theano Version: 1.0.4
* Python Version: 3.7.4
* Operating system: MacOS 10.14.6
* How did you install PyMC3: from source with pip

[ricardoV94]

@rpgoldman Did you ever manage to get a minimal example. Is this still an issue?

[AlexAndorra]

Yeah, I'm not sure it's still an issue actually -- I did some work on pm.Data a few months ago, and it should now be able to be used as input for other random variables (see PR #3925, released in 3.9.0)

[ricardoV94]

@AlexAndorra That's great! It would be really helpful if we could have an example and test it before and after your changes to see if the issue really disappeared with it.

I tried to use it in a model and no obvious errors came up, but I am not sure how @rpgoldman was using it when he found the problem.

[ricardoV94]

@AlexAndorra I checked your PR and I think this unit test you added covers this issue perfectly: https://github.com./pymc-devs/pymc3/blob/03d7af5b6dd5ad99ab2f3bd8ca7987a744dbef46/pymc3/tests/test_data_container.py#L139

I think we can close this one, do you agree?

[AlexAndorra]

Awesome, thanks for checking @ricardoV94 ! Closing 🎉