The Gluster Blog

Gluster blog stories provide high-level spotlights on our users all over the world

A Phyrric methodology for cleaning out hadoop in psuedodistributed mode.

Gluster
2012-03-31

Every once in a while , hadoop goes totally haywire when I play with it in psuedodistributed mode.

Problems include :

1) Data not being replicated to nodes (i.e. you do a namenode format, and the data nodes are now out of sync). 
2) No connection available (i.e. hadoop keeps trying to connect to localhost:9000 and failing) .
3) Other permissions or other types of cryptic exceptions .

The solution is simple – pseudodistributed mode, by default, writes to /tmp (which aliases to /private/tmp on OS X). Thus, to clean up your psuedodistributed hadoop dfs, you can simply :

1) run stop-all.sh (or stop all hadoop services in some other manner).
2) Remove everything in /tmp (Careful here– im assuming you dont have anything important in /tmp – if you do, just remove everything that looks related to hadoop). 
3) hadoop namenode -format : this will format the namenode , starting things over from scratch. 
4) Fire up the rest of the “cluster” by running “start-all.sh”.

BLOG

  • 06 Dec 2020
    Looking back at 2020 – with g...

    2020 has not been a year we would have been able to predict. With a worldwide pandemic and lives thrown out of gear, as we head into 2021, we are thankful that our community and project continued to receive new developers, users and make small gains. For that and a...

    Read more
  • 27 Apr 2020
    Update from the team

    It has been a while since we provided an update to the Gluster community. Across the world various nations, states and localities have put together sets of guidelines around shelter-in-place and quarantine. We request our community members to stay safe, to care for their loved ones, to continue to be...

    Read more
  • 03 Feb 2020
    Building a longer term focus for Gl...

    The initial rounds of conversation around the planning of content for release 8 has helped the project identify one key thing – the need to stagger out features and enhancements over multiple releases. Thus, while release 8 is unlikely to be feature heavy as previous releases, it will be the...

    Read more