What makes RNN deep?
The question is quite simple but indeed has a deep meaning. Networks that have one layer are shallow and those with two or more layers can be considered deep. Moreover RNN are profoundly used in sequence learning based applications and hence the input to the network are long sequences. The data provided is not only complex but also large for conventional neural network to handle. Moreover, the number of units needed are proportionally more. With these large number of units and many hidden layers trying to learn complex data, the network is termed as deep.