Decision Tree - Gini Index

The Gini index is used in CART. The Gini index measures the impurity of the data set,

where p_i - probability that data in the data set, D belong to class, C_iand pi = |C_i,D|/|D|

There are 2^v - 2 possible ways to form two partitions of the data set, D based on a binary split on a attribute.

Each of the possible binary splits of the attribute is considered. The subset that gives the minimum Gini index is selected as the splitting subset for discrete valued attribute.

The degree of Gini index varies between 0 and 1. The value 0 denotes that all elements belong to a certain class or if there exists only one class, and the value 1 denotes that the elements are randomly distributed across various classes. A Gini Index of 0.5 denotes equally distributed elements into some classes.

The Gini index is biased toward multivalued attributes and has difficulty when the number of classes is large.

Exercise 2 - Amdahl's Law

A programmer has parallelized 99% of a program, but there is no value in increasing the problem size, i.e., the program will always be run with the same problem size regardless of the number of processors or cores used. What is the expected speedup on 20 processors? Solution As per Amdahl's law, the speedup, N - No of processors = 20 f - % of parallel operation = 99% = 1 / (1 - 0.99) + (0.99 / 20) = 1 / 0.01 + (0.99 / 20) = 16.807 The expected speedup on 20 processors is 16.807

Gaussian Elimination - Row reduction Algorithm

Gaussian elimination is a method for solving matrix equations of the form, Ax=b. This method is also known as the row reduction algorithm. Back Substitution Solving the last equation for the variable and then work backward into the first equation to solve it. The fundamental idea is to add multiples of one equation to the others in order to eliminate a variable and to continue this process until only one variable is left. Pivot row The row that is used to perform elimination of a variable from other rows is called the pivot row. Example: Solving a linear equation The augmented matrix for the above equation shall be The equation shall be solved using back substitution. The eliminating the first variable (x1) in the first row (Pivot row) by carrying out the row operation. As the second row become zero, the row will be shifted to bottom by carrying out partial pivoting. Now, the second variable (x2) shall be eliminated by carrying out the row operation again. ...

Decision Tree Scalability Methods

The scalable decision tree induction methods are RainForest and BOAT. RainForest method maintains an AVC set for each attribute at each node. AVC stands for Attribute Value Classlabel. BOAT , stands for Bootstrapped Optimistic Algorithm for Tree construction, uses a statistical technique known as bootstrapping., by which several smaller subsets are created. The several trees are created using the subsets and finally the full tree is generated using the trees created by smaller subsets. BOAT was found to be two to three times faster than RainForest.

The Data Ilm

Search This Blog