tensorflow.python.framework.errors_impl.UnknownError: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
2019-08-15 19:35:29.110540: I tensorflow/stream_executor/dso_loader.cc:152] successfully opened CUDA library cublas64_100.dll locally
2019-08-15 19:35:29.371235: E tensorflow/stream_executor/cuda/cuda_blas.cc:510] failed to create cublas handle: CUBLAS_STATUS_ALLOC_FAILED
2019-08-15 19:35:30.351904: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED
2019-08-15 19:35:30.352445: E tensorflow/stream_executor/cuda/cuda_dnn.cc:334] Could not create cudnn handle: CUDNN_STATUS_ALLOC_FAILED
这边的原因是因为gpu被占用了。另一个任务还在跑。停止即可
或者
错误主要指向cudnn，但是CUDA版本和cudnn版本是符合当前tensorflow要求的，因此只能是GPU占用问题导致的。解决方法如下：
tensorflow 框架下设置GPU按需分配：
import tensorflow as tf
config = tf.ConfigProto()
config.gpu_options.allow_growth = True
with tf.Session(config=config) as sess:
…
keras框架（Tensorflow backend） 设置GPU按需分配：
import tensorflow as tf
from keras import backend as K
config = tf.ConfigProto()
config.gpu_options.allow_growth=True
sess = tf.Session(config=config)
K.set_session(sess)
转载自：
https://blog.csdn.net/ghy_111/article/details/86672450
