subject to the condition that d k=1 ak = 1. This property is a special case of a more general theorem involving convex functions (which include the logarithmic function) known as Jensen’s inequality.

Question

By applying Jensen’s inequality, Equation 4.4 can be bounded as follows:

Answer 1

Because E(Y |X) ? E(Y ) ? 0, it follows that entropy never increases after

splitting on an attribute.