using affinity = "full" breaks the code #25
Description
Hi,
Thanks for the paper and the code.
In the networks.py and costs.py you have made an arrangement to deal with case where user wants to calculate full affinity matrix and not just knn affinity matrix but if I set "affinity : full" in run.py, it produces an error related to the cholesky factorization of the output of the feed forward NN.
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/sklearn/externals/joblib/externals/cloudpickle/cloudpickle.py:47: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
Using TensorFlow backend.
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:458: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:459: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:460: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:461: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:462: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/dtypes.py:465: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
2020-12-20 14:18:42.250842: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250902: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250910: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250917: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250923: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX512F instructions, but these are available on your machine and could speed up CPU computations.
2020-12-20 14:18:42.250929: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations.
Epoch: 0, loss=184521.364868, val_loss=12510.053711
Traceback (most recent call last):
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call
return fn(*args)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn
status, run_metadata)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/contextlib.py", line 88, in __exit__
next(self.gen)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status
pywrap_tensorflow.TF_GetCode(status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cholesky decomposition was not successful. The input might not be valid.
[[Node: Cholesky = Cholesky[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](add)]]
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "run.py", line 174, in <module>
x_spectralnet, y_spectralnet = run_net(data, params)
File "/home/suraj/Suraj/newfolder/SpectralNet/src/applications/spectralnet.py", line 94, in run_net
params['spec_ne'])
File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/networks.py", line 166, in train
batches_per_epoch=100)[0]
File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/train.py", line 92, in train_step
return_vars_ += np.asarray(K.get_session().run(all_vars, feed_dict=feed_dict)[:len(return_var)])
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 895, in run
run_metadata_ptr)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1124, in _run
feed_dict_tensor, options, run_metadata)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run
options, run_metadata)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Cholesky decomposition was not successful. The input might not be valid.
[[Node: Cholesky = Cholesky[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](add)]]
Caused by op 'Cholesky', defined at:
File "run.py", line 174, in <module>
x_spectralnet, y_spectralnet = run_net(data, params)
File "/home/suraj/Suraj/newfolder/SpectralNet/src/applications/spectralnet.py", line 89, in run_net
params['n_nbrs'], batch_sizes, siamese_net, x_train, len(x_train_labeled))
File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/networks.py", line 92, in __init__
self.outputs = stack_layers(self.inputs, self.layers)
File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/layer.py", line 104, in stack_layers
l = Orthonorm(outputs['Orthonorm'], name=layer.get('name'));
File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/layer.py", line 39, in Orthonorm
ortho_weights = orthonorm_op(x)
File "/home/suraj/Suraj/newfolder/SpectralNet/src/core/layer.py", line 22, in orthonorm_op
L = tf.cholesky(x_2)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/ops/gen_linalg_ops.py", line 234, in cholesky
result = _op_def_lib.apply_op("Cholesky", input=input, name=name)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/suraj/anaconda3/envs/suraj-tf/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__
self._traceback = self._graph._extract_stack() # pylint: disable=protected-access
InvalidArgumentError (see above for traceback): Cholesky decomposition was not successful. The input might not be valid.
[[Node: Cholesky = Cholesky[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"](add)]]
I am very confused here since cholesky factorization is done on the output (Y^T Y) of the NN and that no way related to the affinity matrix, yet it shows an error that cholesky decomposition was not successful. Ideally, changing "affinity" option should not make any changes to output Y of the NN. Any clarification would be very much helpful!