Customizing the Hadoop Configuration implementation (outside of the cluster)

Dev Code that uses that hadoop needs to be customized. 

If you have control over your dev tests, you can simply use

Configuration.addDefaultResource(“/path/to/core-site.xml”);

This will load the conf file after loading other default resources.

In general, you can expect it to work.

But you might find the need to ensure the right core-site.xml is used *without* writing any java code.

Here’s how to load hadoop Configurations into maven and/or eclipse (yeah, admittedly, this is really just a matter of making sure core-site.xml is on the classpath, so astute eclipse/maven users shouldn’t really need manual instructions here…).

Custom Configuration classes in the eclipse runtime:

Unit tests in eclipse add files to the classpath through the snazzy eclipse ui.

Add the folder containing your core-site.xml to the “source folders on build path”.

Build Path -> Source -> select folder containing core-site.xml.

Custom Configuration classes in your maven builds:

Copy your core-site.xml into the resources/ folder.  This will induce it to be included in tests.  For specifics on how the resources/ directory works, see: http://maven.apache.org/guides/introduction/introduction-to-the-standard-directory-layout.html.