Why is it not typical for the Namenode to be replicated? And how do events like cluster outages get handled with respect to replication and fault tolerance?
@kaiang Wondering about your first question as well. As for the second one, I think losing one rack is fine since we have duplicates for the blocks on other racks. Later in the lecture Kunle also mentioned re-starting the failed jobs.
@ufxela I think it's not very likely to be an issue since the namenode only needs to send metadata, which is much smaller compared to data blocks, to clients and the transfer of data blocks happen only between clients and datanodes.
@kaiang @haiyuem I think one reason is that a single machine is less likely to have a machine failure compared to having multiple namenodes working together, and if the metadata is not large, restarting the namenode can be pretty quick. However, in the HDFS paper (https://storageconference.us/2010/Papers/MSST/Shvachko.pdf), it does mention in chapter II.F that there are backup nodes that stores the same data as the namenode.
Is Namenode here just the Master Node in the previous slide?
Just to tie up this end, yes @wanze it is.
Please log in to leave a comment.
Are there ever any issues with bandwidth on namenode, since its accessed for ever file access, compared to data nodes which aren't.
Also, is this comparable to bit torrent at all?