tensor-flow-basic-syntax2.ipynb

{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Import tensorflow \nimport tensorflow as tf\nprint(\"version: \"+tf.__version__)","metadata":{"_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets see Hello world example\nhello = tf.constant(\"Hello \")\nworld = tf.constant(\"world!\")","metadata":{"_uuid":"beb2fa258893198b3f80669c8d2bddc612cfd362","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets print\nprint(hello+world)","metadata":{"_uuid":"6eb3e8d9003c86b14bbba480d3e92ba58b461412","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets have a look on type of hello and world variables\nprint(type(hello))\nprint(type(world))","metadata":{"_uuid":"110178babf6597e7a6119bed25ddcd09ce4238b2","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"![](http://)In order to print **hello world!** we need tf session","metadata":{"_uuid":"13003de0afe558c956c8ed24d3907d4e451a0d19"}},{"cell_type":"code","source":"with tf.Session() as sess:\n    print( sess.run(hello+world) ) # concatenation","metadata":{"_uuid":"f305cfaeba8b8ce1163e1293d0a000db24ccef4f","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# lets play with numbers\na = tf.constant(5)\nb = tf.constant(10)\ntype(a)","metadata":{"_uuid":"3c29caadfd7bc0127eacdcab68f8dee88296a7d9","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"with tf.Session() as sess:\n    print( sess.run(a+b) )","metadata":{"_uuid":"6531903ea1d5323b77a9303d9556a3c6dfb1daed","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Lets see how matrix will be declared \n    Explore matrix functions by placing curser on tf and (shift+tab)","metadata":{"_uuid":"15885623630006b42632718d18c1fdffaa9cd6bd"}},{"cell_type":"code","source":"# A 4X4 Matrix with all values 10\nmatA = tf.fill((4,4),10) \nwith tf.Session() as sess:\n    print( sess.run(matA) )","metadata":{"_uuid":"6cc11d0ba30625087a2e18bf0a66b359223f38dc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Zero values matrix\nmatB = tf.zeros((2,2))\nwith tf.Session() as sess:\n    print( sess.run(matB) )","metadata":{"_uuid":"a31222ed5dbcf667dcc8a1856a08a70e91aa1708","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Normal distribution Matrix\nmatN = tf.random_normal((2,2),mean=0,stddev=1.0)\n\nwith tf.Session() as sess:\n    print(sess.run(matN))","metadata":{"_uuid":"0b155285e7365e589ccdbf9e39d76f655ca10f3d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Uniform Random Distribution\nmatU = tf.random_uniform((2,2),minval=10,maxval=100)\nwith tf.Session() as sess:\n    print(sess.run(matU))","metadata":{"_uuid":"22e0a7010d46218e82f68aa7f3d18a794b2ec2de","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### To execute tensors we can also create interactive session\n#### tf.InteractiveSession()\nThis is the exact same as tf.Session() but is targeted for using IPython and Jupyter Notebooks that allows you to add things and use Tensor.eval() and Operation.run() instead of having to do Session.run() every time you want something to be computed.","metadata":{"_uuid":"78108abd6528d632de582e01cc727196864eb521"}},{"cell_type":"code","source":"ISess = tf.InteractiveSession()\n\nprint( ISess.run(tf.zeros((5,5)) ) )\n# or\nones = tf.ones((5,5))\nprint( ones.eval() ) \n","metadata":{"_uuid":"a89e2ab7baaccc3632f8e4b54bc99603d2a36a46","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Simple Matrix example\nmatS1 = tf.constant([[10,5],\n                    [3, 9]  ])\nmatS2 = tf.constant([[10,5],\n                    [3, 9]  ])\n\nprint( matS1.eval() )\nprint( matS1.get_shape())\n\nprint(tf.matmul(matS1,matS2).eval())","metadata":{"_uuid":"35f3e1001f42ef9217f08211984bf0280a2ed6d7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Graphs in Tensorflow\nA tf.Graph object defines a namespace for the tf.Operation objects it contains.\n\nA tf.Graph contains two relevant kinds of information:\n\n**Graph structure** The nodes and edges of the graph, indicating how individual operations are composed together, but not prescribing how they should be used. The graph structure is like assembly code: inspecting it can convey some useful information, but it does not contain all of the useful context that source code conveys.\n\n**Graph collections** TensorFlow provides a general mechanism for storing collections of metadata in a tf.Graph. The tf.add_to_collection function enables you to associate a list of objects with a key (where tf.GraphKeys defines some of the standard keys), and tf.get_collection enables you to look up all objects associated with a key. Many parts of the TensorFlow library use this facility: for example, when you create a tf.Variable, it is added by default to collections representing \"global variables\" and \"trainable variables\". When you later come to create a tf.train.Saver or tf.train.Optimizer, the variables in these collections are used as the default arguments.","metadata":{"_uuid":"99099468e8b965e0fb380a3ddd81468c24e10686"}},{"cell_type":"code","source":"tf.get_default_graph","metadata":{"_uuid":"cf157de0065efbe6c62ba1c133f260dc62ada5bc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print( tf.constant(0) )\nprint( tf.constant(0, name=\"c\") )","metadata":{"_uuid":"682896b7b2078215a1afd795c9470746edf4d9f6","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Other than default graph\nd = tf.get_default_graph\ng = tf.Graph()\ng","metadata":{"_uuid":"69678534816e9dee24068c93f2bbf86ee01e4f91","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"g is tf.get_default_graph","metadata":{"_uuid":"cde0f5a64fd9e3eefe7b95e89281224a99f75c54","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"d is tf.get_default_graph","metadata":{"_uuid":"8cd4709e2f165bd484ae768c5058280300d970cc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Variables\nA TensorFlow variable is the best way to represent shared, persistent state manipulated by your program.\nVariables are manipulated via the **tf.Variable class**\n\nVariable needs to be initialized.\n\nCan hold the values of weights and biases through out the session. ","metadata":{"_uuid":"11be80e5d1b62cadfd7ca5093f11467fd75124c8"}},{"cell_type":"code","source":"tensorVar = tf.Variable( initial_value=tf.zeros((2,2)) )\n\n# initialize the variables\ninit = tf.global_variables_initializer()\ninit.run()\n\nprint( tensorVar )\nprint(tensorVar.eval())","metadata":{"_uuid":"225c06f031a81d1bd2ea5a9beedc4f80bc4b9909","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Placeholders\nThey are initially empty and used to feed in the actual training examples. But they need to be declared with datatype and expected shape.","metadata":{"_uuid":"018b2cdc21769a548f1d7cd424739b71f6348754"}},{"cell_type":"code","source":"ph = tf.placeholder(tf.float32) \nph","metadata":{"_uuid":"2ff9affd0b921ab022413ce0def114abb122d438","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"matph1 = tf.placeholder(tf.int32, shape=(3,3))\nmatph2 = tf.placeholder(tf.int32, shape=(None,3))\n# None is for no. of rows or examples in dataset\n\nprint(matph1)\nprint(matph2)","metadata":{"_uuid":"a109dbf32c1a33b5fa34a2183611fdbb34ce0773","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Basic Neural Network with Tensorflow ","metadata":{"_uuid":"634013ff18e2c8ebcf71bb79a1eaf1720ef79177"}},{"cell_type":"code","source":"import numpy as np #for linear algebra \nnp.random.seed(101) # It can be called again to re-seed the generator\ntf.set_random_seed(101)","metadata":{"_uuid":"628da5e587f505061f239d9c8cb29c85391ff1eb","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"rand_a = np.random.uniform(0,100,(5,5))\nrand_b = np.random.uniform(0,100,(5,1))\n\na = tf.placeholder(tf.float32)\nb = tf.placeholder(tf.float32)\n\nadd_op = a+b # tf.add(a,b)\nmult_op = a*b #tf.multiply(a,b)","metadata":{"_uuid":"c6555df9be733160c751521f7c17378c11fbc497","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"**Running Sessions to create Graphs with Feed Dictionaries**","metadata":{"_uuid":"5e203396fd9e8add32fd24e56b616df57206cfdf"}},{"cell_type":"code","source":"with tf.Session() as sess:\n    s = sess.run( add_op, feed_dict={ a:rand_a, b:rand_b} )\n    print(s)\n    \n    m = sess.run( mult_op, feed_dict={a:rand_a, b:rand_b } )\n    print(m)","metadata":{"_uuid":"5d79351307e0002dc9ddecdd4a78502472a48b41","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"n_features = 10\nn_dense_neurons = 3","metadata":{"_uuid":"3faf81118d72342864b78b86a4ea7d74efa65876","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Placeholder for x\nx = tf.placeholder(tf.float32,(None,n_features))\n\n# Variables for w and b\nb = tf.Variable(tf.zeros([n_dense_neurons]))\n\nW = tf.Variable(tf.random_normal([n_features,n_dense_neurons]))","metadata":{"_uuid":"36e797516f01d755cd3cab7e01e1908b9ecff250","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Activation func\nxW = tf.matmul(x,W)\nz = tf.add(xW,b)\n# tf.nn.relu() or tf.tanh()\na = tf.sigmoid(z)\n\n# Variable Intialization\ninit = tf.global_variables_initializer()\nwith tf.Session() as sess:\n    sess.run(init)\n    print(sess.run(z,feed_dict={x : np.random.random([1,n_features])}))\n    layer_out = sess.run(a,feed_dict={x : np.random.random([1,n_features])})\n\nprint(layer_out)","metadata":{"_uuid":"87969218b08e66b074b6278412aab44827addfab","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Full Neural Network Example\n**${y = mx + b}$**\n","metadata":{"_uuid":"ec10193e4bb698e573e9452544a19dda40dd1f4c"}},{"cell_type":"code","source":"np.linspace(1,10,10)","metadata":{"_uuid":"2510123c018a23ae29204434d78e59d0834fe4f7","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Artificial Data\nxdata = np.linspace(0,10,10) + np.random.uniform(-1.5,1.5,10)\nprint(xdata)\nydata = np.linspace(0,10,10) + np.random.uniform(-1.5,1.5,10)\nprint(ydata)","metadata":{"_uuid":"4d7841b63d991c359951ddc723f68bda44c8a752","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"m = tf.Variable(0.39)\nb = tf.Variable(0.2)","metadata":{"_uuid":"939da4c141ed360b9ab234a87920ca9e49552c3d","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Cost or Residual \n\nerror = 0\nfor x,y in zip(xdata,ydata):\n    y_ = m*x + b\n    error += (y-y_)**2\n\nprint(error)","metadata":{"_uuid":"1ed95a3a4306b6351241af84466657ffa047f2dc","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# optimizer\noptimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001)\ntrain = optimizer.minimize(error)\ntrain","metadata":{"_uuid":"10f465206444d933f60f4b24d8dab5ee90bb8e29","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"init = tf.global_variables_initializer()\n\nwith tf.Session() as sess:\n    \n    sess.run(init)\n    \n    epochs = 100\n    \n    for i in range(epochs):\n        \n        sess.run(train)\n        \n\n    # Fetch Back Results\n    final_slope , final_intercept = sess.run([m,b])","metadata":{"_uuid":"e9294c54675005fcab0d769350780387a40fd8c2","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(final_slope)\nprint(final_intercept)","metadata":{"_uuid":"b9dc2532641be3ac164c2c81bec3644ba4563b80","trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import matplotlib.pyplot as plt\n\nx_test = np.linspace(-1,11,10)\ny_pred_plot = final_slope*x_test + final_intercept\n\nplt.plot(x_test,y_pred_plot,'r')\n\nplt.plot(xdata,ydata,'*')\nplt.show()","metadata":{"_uuid":"d8459f99bb6695d2fa6f25180ba837d85fd13dbd","trusted":true},"execution_count":null,"outputs":[]}]}