These examples are the same as those explained using the C wrapper, but in this case, the C++ API is directly used.
We will start with a simple appplication skeleton which does nothing.
#include <iostream> int main(int argc, char *argv[]) { return 0; }
Now, we want to start using DMLite, so first of all we need to include the headers and
instantiate a dmlite::PluginManager
,
which controls the loading and instantiation process of a plugin stack. Then, we will request the manager
to use the configuration file specified as the first parameter of the application.
#include <iostream> #include <dmlite/cpp/dmlite.h> int main(int argc, char *argv[]) { dmlite::PluginManager manager; if (argc < 2) { std::cout << "Need at least one parameter." std::endl; return 1; } try { manager.loadConfiguration(argv[1]); } catch (dmlite::DmException& e) { std::cout << "Could not load the configuration file." << std::endl << "Reason: " << e.what() << std::endl; return 1; } return 0; }
Now you can compile and try it.
# gcc main.cpp -ldmlite -o dmlite-init # ./dmlite-init /etc/init.conf
To actually start to perform actions on the namespace, we need to instantiate a
dmlite::StackInstance
before, and
to set a security context, either directly, or passing some credentials first.
#include <iostream> #include <dmlite/cpp/authn.h> #include <dmlite/cpp/dmlite.h> #include <dmlite/cpp/catalog.h> int main(int argc, char *argv[]) { dmlite::PluginManager manager; if (argc < 3) { std::cout << "Usage: " << argv[0] << " <configuration file> <path>" << std::endl; return 1; } // Create manager try { manager.loadConfiguration(argv[1]); } catch (dmlite::DmException& e) { std::cout << "Could not load the configuration file." << std::endl << "Reason: " << e.what() << std::endl; return 2; } // Create StackInstance dmlite::StackInstance stack(&manager); // Set security credentials dmlite::SecurityCredentials creds; creds.clientName = "/C=CH/O=CERN/OU=GD/CN=Test user 1"; creds.remoteAddress = "127.0.0.1"; try { stack.setSecurityCredentials(creds); } catch (dmlite::DmException& e) { std::cout << "Could not set the credentials." << std::endl << "Reason: " << e.what() << std::endl; return 4; } // Action // Done return 0; }
Please, note that the client name you are setting must be defined in the file /etc/lcgdm-mapfile, so a default group can be picked. You can always initialize the field fqans to contain at least one group (i.e. dteam)
One advantage of the C++ API is that you don't need to take care of freeing resources everytime an error occurs. Since PluginManager and StackInstance can be used as objects, and not as pointers, they will be automatically destroyed as soon as they fall out of scope.
Let's add the code that is going to actually do something.
// Action dmlite::Catalog* catalog = stack.getCatalog(); try { dmlite::ExtendedStat xstat = catalog->extendedStat(argv[2]); std::cout << "File name: " << xstat.name << std::endl << "File owner: " << xstat.stat.st_uid << std::endl << "File group: " << xstat.stat.st_gid << std::endl << "File mode: " << std::oct << xstat.stat.st_mode << std::endl; } catch (dmlite::DmException& e) { std::cout << "Could not stat the file." << std::endl << "Reason: " << e.what() << std::endl; return e.code(); }
You should get this output now:
# ./dmlite-init /etc/dmlite.conf / File name: / File owner: 0 File group: 0 File mode: 40755 # ./dmlite-init /etc/dmlite.conf /xx Could not stat the file. Reason: [#00.000002] Entry 'xx' not found under ''
For this example, we will reuse the base code as shown just before.
// Action dmlite::Catalog* catalog = stack.getCatalog(); try { dmlite::Directory* dir = catalog->openDir(argv[2]); dmlite::ExtendedStat* xstat; std::cout << "Content of the directory " << argv[2] << std::endl; while ((xstat = catalog->readDirx(dir)) != NULL) { std::cout << '\t' << xstat->name << std::endl; } catalog->closeDir(dir); } catch (dmlite::DmException& e) { std::cout << "Could not open the directory." << std::endl << "Reason: " << e.what() << std::endl; return e.code(); }
The output this time should be:
# ./dmlite-init /etc/dmlite.conf /xx Could not open the directory. Reason: [#00.000002] Entry 'xx' not found under '' # ./dmlite-init /etc/dmlite.conf / Content of the directory / dpm
This use case is a little bit trickier, since replicas do not necessarily point to a physical file somewhere. It may be the case, but you should not rely on it, since some translation may be needed (i.e. for Hadoop or S3 pools).
For this example, you will need to include the header dmlite/cpp/poolmanager.h
dmlite::PoolManager* poolManager = stack.getPoolManager(); try { dmlite::Location location = poolManager->whereToRead(argv[2]); dmlite::Location::const_iterator i; int n; for (i = location.begin(), n = 0; i != location.end(); ++i, ++n) { std::cout << "Chunk " << n << ": " << i->host << ':' << i->path << " (" << i->offset << '-' << i->offset + i->size << ")" << std::endl; std::vector<std::string> keys = i->getKeys(); std::vector<std::string>::const_iterator j; for (j = keys.begin(); j != keys.end(); ++j) { std::cout << '\t' << *j << ": " << i->getString(*j) << std::endl; } } } catch (dmlite::DmException& e) { std::cout << "Could not get the final location for " << argv[2] << std::endl << "Reason: " << e.what() << std::endl; return e.code(); }
The output will depend on what you have on your DPM server, but the output should look something like this:
# ./dmlite-init /etc/dmlite.conf /dpm/cern.ch/home/dteam/file.html Chunk 0: sl6vm.cern.ch:/dpm/cern.ch/home/dteam/dmlite03_test/test529.html (0-63214) token: NwwcFOCZDUjPCccmrNulbyckWTQ=@1348067073@0
Now, this gives the physical location of the replica, but it is not tied to any protocol in particular. The front-end is the responsible of determining which protocol should be used to access the actual data.
For writing, whereToWrite
works in a similar manner, but a call to doneWriting
must be done when the writing finishes.
There is one limitation of DMLite design, which is that only one I/O plugin can be used at a time. This is because in principle each node belongs to one single pool. This is, one node can not belong to a Hadoop pool and to a Filesytem pool at the same time.
To open a file four things are needed: a stack instance, a physical path, an open mode (read, write, ...) and whatever extra information
the corresponding pool plugin gave us. For instance, in the previous example we show that extra information, stored as key/value pairs. Usually
this data includes the necessary information to validate the access (i.e. a token).
Once the file is opened, usual read and/or write operations (and also seek/tell/eof/...) can be performed.
The following example assumes you have already deserialized the additional information coming from the head node, and put it into an
dmlite::Extensible
object.
#include <dmlite/cpp/io.h> #include <iostream> int dump_data(dmlite::StackInstance& stack, const char *path, const dmlite::Extensible& args, std::ostream& out) { char buffer[1024]; size_t size; dmlite::IODriver* iodriver; iodriver = stack.getIODriver(); try { dmlite::IOHandler* in = iodriver->createIOHandler(path, O_RDONLY, args) while (!in->eof()) { size = in->read(buffer, sizeof(buffer)); out.write(buffer, size); } delete in; } catch (dmlite::DmException& e) { std::cout << "Could not open the input file: " << path << std::endl; std::cout << "Reason: " << e.what() << std::endl; return e.code(); } return 0; }