How to separate Hadoop MapReduce from HDFS? -


i'm curious if separate hdfs filesystem mapreduce framework. know main point of hadoop run maps , reduces on machines data in question, wondering if change *.xml files change configuration of machine jobtracker, namenode , datanodes running on.

currently, configuration 2 vms setup: 1 (the master) namenode, datanode, jobtracker, tasktracker (and secondarynamenode), other (the slave) datanode, tasktraker. essentially, want change have master namenode datanode(s), jobtracker, , have slave tasktracker perform computations (and later on, have more slaves tasktrackers on them; 1 on each). bottleneck data transfer between 2 vms computations of maps , reduces, since data @ stage small i'm not concerned it. know if configuration possible, , how it. tips?

thanks!

you don't specify kind of options in configuration files. have take care of kind of deamons start on each machine(you call them vms think mean machines).

i suppose start using start-all.sh script can find in bin directory under hadoop installation dir.

if take @ script see call number of sub-scripts corresponding starting datanodes, tasktrackers , namenode, jobtracker.

in order achive you've said, this:

  1. modify masters , slaves files this: master file should contain name of machine1 slaves should contain name of machine2

  2. run start-mapred.sh

  3. modify masters , slaves files this: master file should contain machine1 slaves file should contain machine1

  4. run start-dfs.sh

i have tell i've never tried such configuration i'm not sure going work can give try. anyway solution in direction!


Comments

Post a Comment

Popular posts from this blog

JQuery Autocomplete without using label, value, id -

is possible have jquery autocomplete work without using conventional id,label,value? for instance, not wish use: [ { "id": "botaurus stellaris", "label": "great bittern", "value": "great bittern" }, { "id": "asio flammeus", "label": "short-eared owl", "value": "short-eared owl" }] but rather: [ { "my_id": "botaurus stellaris", "first_name": "great bittern", "last_name": "great bittern" }, { "my_id": "asio flammeus", "first_name": "short-eared owl", "last_name": "short-eared owl" }] so like, jquery autocomplete takes id,value,label , can make take my_id, first_name, last_name , make jquery autocomplete treat same id,label,value ? this might useful, when don't wish modify data's keys coming datasource, display on jquery autocomplete. ...
Read more

c++ - Accessing inactive union member and undefined behavior? -

i under impression accessing union member other last 1 set ub, can't seem find solid reference (other answers claiming it's ub without support standard). so, undefined behavior? the confusion c explicitly permits type-punning through union, whereas c++ ( c++11 ) has no such permission. c11 6.5.2.3 structure , union members 95) if member used read contents of union object not same member last used store value in object, appropriate part of object representation of value reinterpreted object representation in new type described in 6.2.6 (a process called ‘‘type punning’’). might trap representation. the situation c++: c++11 9.5 unions [class.union] in union, @ 1 of non-static data members can active @ time, is, value of @ 1 of non-static data members can stored in union @ time. c++ later has language permitting use of unions containing struct s common initial sequences; doesn't permit type-punning. to determine whether union type-punning is allowe...
Read more

JAVA - what is the difference between void and boolean methods? -

i new java. writing wrapper-library in java make functions available in basic-like language. i got stock @ point when noted code not executed in java-library although compiler did not complain (using eclipse). resolved replacing code follows: public void videoquality(int vquality) //did not work into public boolean videoquality(int vquality) //works here complete code-snippets: public void videoquality(int vquality) //did not work {if (vquality==16) { vidquality=16; } else if (vquality==-16) { vidquality=-16; } else if (vquality==0) { vidquality=0; } else vidquality=-16; vitamioext.setvideoquality(vidquality); } public boolean videoquality(int vquality) //works {if (vquality==16) { vidquality=16; } else if (vquality==-16) { vidquality=-16; } else if (vquality==0) { vidquality=0; } else vidquality=-16; vitamioext.setvideoquality(vidquality); return true; } i think void corresponds sub in visual basic while boolean corresponds function. i found odd following code worked using...
Read more