Webhdfs (client side only), does not affect other parts of hdfs or cause dataloss.
Make a “create” call using rest api (hdfs api) and send an invalid value for parameter overwrite
The hdfs needs to be enabled.
1. Enable webhdfs by add this property to the configuration file
2. Start hdfs
3. make a create call using rest api and send an invalid value for parameter overwrite
4. observe returned content type: type text/html; charset=utf-8
Hdfs with webhdfs must be started before the error can be reproduced. But this is trivial.
Since this failure is observed on the webhdfs client side, it is very hard to diagnose. There are no logs on the webhdfs client side. Logs on the server side is not helpfu.
Domain knowledge is needed for this failure. Developers need to know where the bug is. Even then the developer took more than 20 increment to fix this “simple problem” part of a much larger problem.
The symptom is the wrong content type (text/html) is returned instead of application/json type.
Backward inference is virtually impossible. But a rough guideline is outlined below.
1. HDFS-2460 returns unexpected content type.
2. Realize that it seems similar to an earlier bug (HDFS-2453) had 2 content types being returned.
3. So we look at hdfs-2453 instead which is about “tail using a webhdfds uri throws an error”. At first, the description is completely useless. Only digging into the details would one know that this two failures are somewhat related. Infact hdfs-2460 is a special case of hdfs-2453.
4. However upon looking at hdfs-2453, it is not intuitive. It incorporates another failure hdfs-2456) as well being affected by HDFS-2385. HDFS-2385 depends on MAPREDUCE-2764 and HADOOP-7510.
5. So to summarize, backward inference for hdfs2460 is only possible if someone has worked on HDFS-2453, hdfs 2456, hdfs-2385, mapreduce-2764, hadoop-7510.
Semantic error (no “error handling” for error case, only the “success” case is handled). See HDFS-2453
Apply HDFS-2453 patch. We do not “fix” the bug. We added basically whole functions to implement the missing “feature”. Code can be found in hdfs-2453 latest patch file.
Write new functions to allow webhdfs to return the correct content type. Or run the patch.