Lab09-1. NN for XOR

NN for XOR

1. XOR 문제 해결

• XOR

  

  
  

1) Logistic regression으로 문제 해결하기: 해결 불가능

• 소스코드

  import tensorflow as tf
  import numpy as np
  
  x_data = np.array([[0,0], [0,1], [1,0], [1,1]], dtype=np.float32)
  y_data = np.array([[0], [1], [1], [0]], dtype=np.float32)
  
  X = tf.placeholder(tf.float32)
  Y = tf.placeholder(tf.float32)
  W = tf.Variable(tf.random_normal([2, 1]), name='weight')
  b = tf.Variable(tf.random_normal([1]), name='bias')
  
  # hypothesis using sigmoid
  hypothesis = tf.sigmoid(tf.matmul(X, W) + b)
  
  # cost/loss function
  cost = -tf.reduce_mean(Y * tf.log(hypothesis) + (1 - Y) * tf.log(1 - hypothesis))
  train = tf.train.GradientDescentOptimizer(learning_rate=0.1).minimize(cost)
  
  predicted = tf.cast(hypothesis > 0.5, dtype=tf.float32)
  accuracy = tf.reduce_mean(tf.cast(tf.equal(predicted, Y), dtype=tf.float32))
  
  # lanunch graph
  with tf.Session() as sess:
      sess.run(tf.global_variables_initializer())
  
      for step in range(10001):
          sess.run(train, feed_dict={X: x_data, Y: y_data})
  
          if step % 100 == 0:
              print("Step: ", step, "Cost: ", sess.run(cost, feed_dict={X: x_data, Y: y_data}), sess.run(W))
  
      # accuracy report
      h, c, a = sess.run([hypothesis, predicted, accuracy], feed_dict={X: x_data, Y: y_data})
      print("\nHypothesis: ", h, "\nCorrect: ", c, "\nAccuracy: ", a)
  

   

• 결과

  • 해결불가능 Accuracy가 낮음

  Hypothesis:  [[0.5]
   [0.5]
   [0.5]
   [0.5]] 
  Correct:  [[0.]
   [0.]
   [0.]
   [0.]] 
  Accuracy:  0.5
  

 

2. Neural nets으로 XOR문제 해결하기

• Neural nets이 더 deep, wide 해질 수 록 예측이 더 정확해짐

1) Neural nets으로 문제 해결하기: 해결가능

• 소스코드

  (...)
  X = tf.placeholder(tf.float32, [None, 2])
  Y = tf.placeholder(tf.float32, [None, 1])
  
  W1 = tf.Variable(tf.random_normal([2, 2]), name='weight1')
  b1 = tf.Variable(tf.random_normal([2]), name='bias1')
  layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)
  
  W2 = tf.Variable(tf.random_normal([2, 1]), name='weight2')
  b2 = tf.Variable(tf.random_normal([1]), name='bias2')
  hypothesis = tf.sigmoid(tf.matmul(layer1, W2) + b2)
  (...)
  

   

• 결과

  Hypothesis:  [[0.01397681]
   [0.9804833 ]
   [0.9876809 ]
   [0.01199236]] 
  Correct:  [[0.]
   [1.]
   [1.]
   [0.]] 
  Accuracy:  1.0
  

 

2) Deep neural nets

• 소스코드

  (...)
  X = tf.placeholder(tf.float32, [None, 2])
  Y = tf.placeholder(tf.float32, [None, 1])
  
  W1 = tf.Variable(tf.random_normal([2, 10]), name='weight1')
  b1 = tf.Variable(tf.random_normal([10]), name='bias1')
  layer1 = tf.sigmoid(tf.matmul(X, W1) + b1)
  
  W2 = tf.Variable(tf.random_normal([10, 10]), name='weight2')
  b2 = tf.Variable(tf.random_normal([10]), name='bias2')
  layer2 = tf.sigmoid(tf.matmul(layer1, W2) + b2)
  
  W3 = tf.Variable(tf.random_normal([10, 10]), name='weight3')
  b3 = tf.Variable(tf.random_normal([10]), name='bias3')
  layer3 = tf.sigmoid(tf.matmul(layer2, W3) + b3)
  
  W4 = tf.Variable(tf.random_normal([10, 1]), name='weight4')
  b4 = tf.Variable(tf.random_normal([1]), name='bias4')
  hypothesis = tf.sigmoid(tf.matmul(layer3, W4) + b4)
  (...)
  

   

• 결과

  Hypothesis:  [[0.00126288]
   [0.9987746 ]
   [0.9986027 ]
   [0.00179633]] 
  Correct:  [[0.]
   [1.]
   [1.]
   [0.]] 
  Accuracy:  1.0
  

 

3. Wide and Deep NN for MNIST

• MNIST에 wide, deep NN을 적용해보았으나 Accuracy가 떨어지는 문제 발생