tensorflow가있는 AUC를 계산하는 방법은 무엇입니까?

Question

저는 Tensorflow를 사용하여 이진 분류자를 만들었습니다. 이제 AUC와 정확도를 사용하여 분류자를 평가하고 싶습니다.

지금까지 정확도에 관한 한, 나는 쉽게 다음과 같이 수행 할 수 있습니다

AUC를 계산할 때

X = tf.placeholder('float', [None, n_input]) 
y = tf.placeholder('float', [None, n_classes]) 
pred = mlp(X, weights, biases, dropout_keep_prob) 
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1)) 
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) 

나는 다음과 같은 사용

print(tf.argmax(pred, 1).dtype.name) 
print(tf.argmax(pred, 1).dtype.name) 

a = tf.cast(tf.argmax(pred, 1),tf.float32) 
b = tf.cast(tf.argmax(y,1),tf.float32) 

auc = tf.contrib.metrics.streaming_auc(a, b) 

및 교육 루프 :

train_acc = sess.run(accuracy, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.}) 
train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.}) 

다음 출력 (및 오류) 오류가 발생합니다.

int64 
int64 
/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/array_ops.py:1197: VisibleDeprecationWarning: converting an array with ndim > 0 to an index will result in an error in the future 
    result_shape.insert(dim, 1) 
Net built successfully... 

Starting training... 

Epoch: 000/300 cost: 0.618990561 
Traceback (most recent call last): 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 715, in _do_call 
    return fn(*args) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 697, in _run_fn 
    status, run_metadata) 
    File "/usr/lib/python3.5/contextlib.py", line 66, in __exit__ 
    next(self.gen) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/errors.py", line 450, in raise_exception_on_not_ok_status 
    pywrap_tensorflow.TF_GetCode(status)) 
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives 
    [[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]] 

During handling of the above exception, another exception occurred: 

Traceback (most recent call last): 
    File "./mlp_.py", line 152, in <module> 
    train_auc = sess.run(auc, feed_dict={X: batch_xs, y: batch_ys, dropout_keep_prob:1.}) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 372, in run 
    run_metadata_ptr) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 636, in _run 
    feed_dict_string, options, run_metadata) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 708, in _do_run 
    target_list, options, run_metadata) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/client/session.py", line 728, in _do_call 
    raise type(e)(node_def, op, message) 
tensorflow.python.framework.errors.FailedPreconditionError: Attempting to use uninitialized value auc/false_positives 
    [[Node: auc/false_positives/read = Identity[T=DT_FLOAT, _class=["loc:@auc/false_positives"], _device="/job:localhost/replica:0/task:0/cpu:0"](auc/false_positives)]] 
Caused by op 'auc/false_positives/read', defined at: 
    File "./mlp_.py", line 121, in <module> 
    auc = tf.contrib.metrics.streaming_auc(a, b) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 718, in streaming_auc 
    predictions, labels, thresholds, ignore_mask) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 603, in _tp_fn_tn_fp 
    false_positives = _create_local('false_positives', shape=[num_thresholds]) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/contrib/metrics/python/ops/metric_ops.py", line 75, in _create_local 
    collections=collections) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 211, in __init__ 
    dtype=dtype) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/variables.py", line 319, in _init_from_args 
    self._snapshot = array_ops.identity(self._variable, name="read") 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_array_ops.py", line 831, in identity 
    result = _op_def_lib.apply_op("Identity", input=input, name=name) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/op_def_library.py", line 704, in apply_op 
    op_def=op_def) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 2260, in create_op 
    original_op=self._default_original_op, op_def=op_def) 
    File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py", line 1230, in __init__ 
    self._traceback = _extract_stack() 

내가 잘못하고있는 이유와 정확도만을 사용할 때 코드가 잘 돌아가는 것을 이해하지 못한다. 그러나 AUC를 사용할 때이 오류가 발생한다. 이 문제를 해결하는 방법을 이해하려면 올바른 방향으로 나에게 힌트를 주시기 바랍니다.

내 목적은 이진 분류기 성능을 더 잘 평가할 수 있도록 AUC 및 ROC를 계산하는 것입니다.

Answer 1

나는 github에서 같은 문제를 발견했습니다. 현재 tf.contrib.metrics.streaming_auc()을 작동 시키려면 sess.run(tf.initialize_local_variables())을 실행해야합니다. 그들은 그것에 대해 연구하고 있습니다. 여기

당신이이 문제를 해결할 수있는 방법을 설명하는 예제가 있습니다

import tensorflow as tf 

a = tf.Variable([0.1, 0.5]) 
b = tf.Variable([0.2, 0.6]) 

auc = tf.contrib.metrics.streaming_auc(a, b) 

sess = tf.Session() 
sess.run(tf.initialize_all_variables()) 
sess.run(tf.initialize_local_variables()) # try commenting this line and you'll get the error 
train_auc = sess.run(auc) 

print(train_auc)