Raymii.org
Quis custodiet ipsos custodes?Home | About | All pages | Cluster Status | RSS Feed
Execute a command and get both output and exit status in C++ (Windows & Linux)
Published: 07-06-2021 | Author: Remy van Elst | Text only version of this article
❗ This post is over three years old. It may no longer be up to date. Opinions may have changed.
Table of Contents
Recently I had to parse some command line output inside a C++ program. Executing a command and getting just the exit status is easy using std::system, but also getting output is a bit harder and OS specific. By using popen, a POSIX C function we can get both the exit status as well as the output of a given command. On Windows I'm using _popen, so the code should be cross platform, except for the exit status on Windows is alway 0, that concept does not exist there. This article starts off with a stack overflow example to get just the output of a command and builds on that to a safer version (null-byte handling) that returns both the exit status as well as the command output. It also involves a lot of detail on fread vs fgets and how to handle binary data.
Recently I removed all Google Ads from this site due to their invasive tracking, as well as Google Analytics. Please, if you found this content useful, consider a small donation using any of the options below:
I'm developing an open source monitoring app called Leaf Node Monitoring, for windows, linux & android. Go check it out!
Consider sponsoring me on Github. It means the world to me if you show your appreciation and you'll help pay the server costs.
You can also sponsor me by getting a Digital Ocean VPS. With this referral link you'll get $200 credit for 60 days. Spend $25 after your credit expires and I'll get $25!
The complete code example with usage examples can be found on github here or at the bottom of this page. A working example is compiled on github actions for different platforms (windows & linux).
Normally I would advise against parsing command line output. It is error-prone,
you're dependent on the language selected by the user, different versions might
have different flags (OS X vs Linux) and much more. If you have the option
to use a native library, you should use that. An example could be parsing curl
output to get some data from an API. There are probably a metric ton of http
libraries available for your favorite programming language to use instead of
parsing the curl or wget or fetch output.
In my case I have to use an old program to parse a closed-source file to get some binary output. This is a temporary situation, a native parsing library is also under development. The binary is under my control as well as the system settings, language, other tools and such, so for this specific use case the solution to parse command line output was acceptable for the time being.
Do note that in this post I'll interchange the term nullbyte, null character,
null termination and null-terminated. They all mean the same, the null-byte
character used to end a C string (\0, or ^@, U+0000 or 0x00, you
get the gist).
If you need more features, more cross-platform or async execution, boost.Process is a great alternative. I however can't use boost on the environment this code is going to run due to compiler and size constraints.
The stackoverflow example using fgets
On stackoverflow the example given is a good base to build on, however,
to get the exit code and the output, it must be modified. Because we want to
also grab the exit code, we cannot use the example which uses the
std::unique_ptr. Which in itself is a great example of using a unique_ptr with
a custom deleter (cmd is a const char* with the command to execute:
std::unique_ptr<FILE, decltype(&pclose)> pipe(popen(cmd, "r"), pclose);
The code is copied below:
std::string exec(const char* cmd) {
char buffer[128];
std::string result = "";
FILE* pipe = popen(cmd, "r");
if (!pipe) throw std::runtime_error("popen() failed!");
try {
while (fgets(buffer, sizeof buffer, pipe) != NULL)
result += buffer;
}
} catch (...) {
pclose(pipe);
throw;
}
pclose(pipe);
return result;
}
This example does what it states, but with a few gotchas. It uses a FILE*
(pointer), char buffer allocation and manually closing the FILE* when
something goes wrong (catch). The unique_ptr example is more modern, due
to not having to handle the exception and using a std::array<char, 128>
instead of a C-style char* buffer. Exception throwing is a whole other
issue, but let's not get into that today. What we are going to get into
today is C style code to read from FILE* and how binary data is handled.
The stackoverflow example is probably just fine if you only need textual output
into a std::string. My usecase however was a bit more complex, as you'll
find out while reading the rest of this artice.
fread vs fgets
Code style aside, my biggest issue was that using fgets in this way combined
with adding a const char* to a std::string stops when it encounters a
nullyte (\0). For regular string output that often is not an issue, most
commands just output a few strings and call it a day. My output returns a
binary blob, which might include nullbytes. fread reads an amount of bytes
and returns how much it has read successfully, which we can use when
adding the output to our std::string including the nullbytes.
The example above does result += buffer, adding a const char* to a std::string,
in this case according to cppreference on operator+= on std::string:
Appends the null-terminated character string pointed to by s.
The problem therein lies that the characters after the nullbyte should be
added as well in my case. fgets does not give back the amount of data it
read. Using fgets and a buffer of 128, if I have a nullbyte at 10 and a
newline at 40, then the first 10 bytes plus what is after 40 bytes will be
returned. Effectively we're loosing everything in between the nullbyte and the
newline, or until the end of the buffer (128) if there is no newline in
between.
fread does return the amount of bytes it has read. Combining that with a
constructor of std::string that takes a const char* and a size_t we can
force the entire contents inside the string. This is safe, since a
std::string knows its size, it does not rely on a null-termination
character. However, other code that uses const char* will not be able
to work with these nullbytes, keep that in mind.
This stackoverflow post was very helpful for me to understand fread,
as well as help from a co-worker that dreams in C, he explained a lot
of the inner workings.
And if, after all of this, you're wondering why I'm shoehorning binary data
inside a std::string, great question. I'll probably go into that another
time since that would require a longer post than this entire article.
Command execution including output and exit code
My code checks the exit status of the executed binary (for error handling) and
uses the data returned for further processing. To keep this all in one handy
dandy place, lets start with defining a struct to hold that data. It will hold
the result of a command, so the name CommandResult sounds descriptive enough.
Below you'll find the struct code, including an equality operator as well as a stream output operator.
struct CommandResult {
std::string output;
int exitstatus;
friend std::ostream &operator<<(std::ostream &os, const CommandResult &result) {
os << "command exitstatus: " << result.exitstatus << " output: " << result.output;
return os;
}
bool operator==(const CommandResult &rhs) const {
return output == rhs.output &&
exitstatus == rhs.exitstatus;
}
bool operator!=(const CommandResult &rhs) const {
return !(rhs == *this);
}
};
The meat and potatoes of the struct are of course the output and exitstatus. I'm
using an int for the exit status because of reasons.
The next part is the Command class itself, here is that code:
class Command {
public:
/**
* Execute system command and get STDOUT result.
* Like system() but gives back exit status and stdout.
* @param command system command to execute
* @return CommandResult containing STDOUT (not stderr) output & exitstatus
* of command. Empty if command failed (or has no output). If you want stderr,
* use shell redirection (2&>1).
*/
static CommandResult exec(const std::string &command) {
int exitcode = 255;
std::array<char, 1048576> buffer {};
std::string result;
#ifdef _WIN32
#define popen _popen
#define pclose _pclose
#define WEXITSTATUS
#endif
FILE *pipe = popen(command.c_str(), "r");
if (pipe == nullptr) {
throw std::runtime_error("popen() failed!");
}
try {
std::size_t bytesread;
while ((bytesread = fread(buffer.data(), sizeof(buffer.at(0)), sizeof(buffer), pipe)) != 0) {
result += std::string(buffer.data(), bytesread);
}
} catch (...) {
pclose(pipe);
throw;
}
exitcode = WEXITSTATUS(pclose(pipe));
return CommandResult{result, exitcode};
}
}
The fread command will run until there are no more bytes returned from the
command output. I know the kind of output I'm working with so my buffer is 1
MiB, which is probably too large for your data. In my case I benchmarked it
and between 10KiB and 1 MiB was the fastest on the target architecture. 128 or
8192 is just fine as well probably, but you should benchmark that for
yourself. A rather simple test is to output some enormous file with cat and
take the execution time plus cpu and memory usage. Don't print the result,
just look at those three things and choose what ratio is acceptable for you.
Why not also initialize the std::string with 1 MiB of characters? std::strings
cannot be allocated for a given size at construction, other than by filling them or
afterwards calling .reserve(), my benchmarks did not show any meaningful speed
or performance boost by doing either.
Using the above code is easy. Since it's a static function, you don't need a class instance to use it. Here is an example:
std::cout << Command::exec("echo 'Hello you absolute legends!'") << std::endl;
Which results in:
command exitstatus: 0 output: Hello you absolute legends!
Since we're going through a shell, redirection works as well. Redirecting stdout
to stderr results in no output, just an exit status:
std::cout << Command::exec("echo 'Hello you absolute legends!' 1>&2") << std::endl;
The output is on stderr in my shell however, which is expected:
If you do need to capture stderr then you redirect output the other way around, like
so:
std::cout << Command::exec("/bin/bash --invalid 2>&1") << std::endl;
Pipes work as well as in your shell, but do note that this is all using sh and you
have no control over environment variables or the default shell. Read more on the
POSIX page on popen to find out why that is.
A note on Windows
Here is an example for Windows, where we must use _popen and _pclose:
std::cout << "Windows example:" << std::endl;
std::cout << Command::exec("dir * /on /p") << std::endl;
The exit code will always be zero since that concept does not translate to
windows. There is %ErrorLevel%, but that is only an environment variable
for console applications, not the actual exit status.
The microsoft page also notes that _popen will not work with GUI applications,
just console programs. If you need that, use Boost.process or system.
Nullbytes in output example:
In the example code on github you'll also see a execFgets function, I've left that
in there to show the difference in nullbyte handling. For reference I'll show an
example here as well. The relevant part of command using fgets:
while (std::fgets(buffer.data(), buffer.size(), pipe) != nullptr)
result += buffer.data();
The part using fread:
std::size_t bytesread;
while ((bytesread = fread(buffer.data(), sizeof(buffer.at(0)), sizeof(buffer), pipe)) != 0)
result += std::string(buffer.data(), bytesread);
The test command, including a clang-tidy warning exclusion (// NOLINT):
int main() {
using namespace raymii;
std::string expectedOutput("test\000abc\n", 9); //NOLINT
commandResult nullbyteCommand = command::exec("/usr/bin/printf 'test\\000abc\\n'"); // NOLINT(bugprone-string-literal-with-embedded-nul)
commandResult fgetsNullbyteCommand = command::execFgets("/usr/bin/printf 'test\\000abc\\n'"); // NOLINT(bugprone-string-literal-with-embedded-nul)
std::cout << "Expected output: " << expectedOutput << std::endl;
std::cout << "Output using fread: " << nullbyteCommand << std::endl;
std::cout << "Output using fgets: " << fgetsNullbyteCommand << std::endl;
return 0;
}
Output:
Expected output: test\0abc
A command with nullbytes using fread: exitstatus: 0 output: test\0abc
A command with nullbytes using fgets: exitstatus: 0 output: test
The nullbyte character is substituted with \0 in the above output. Here is a screenshot showing
how it looks in my terminal:
Once again do note that this is safe to use with std::strings, methods that take a string_view or
a const char* probably will not react very well to nullbytes. For my use case this is safe, your
milage may vary.
Try playing with the buffer size and then looking at the output. If you set it to 4, the output
with fgets is testbc. Funny right? I like such things.
Complete code
Below you can find the header file command.h. It is also on my github. If you want
usage examples you can find them in the github project main.cpp file.
#command.h
#ifndef COMMAND_H
#define COMMAND_H
// Copyright (C) 2021 Remy van Elst
//
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU General Public License for more details.
//
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
#include <array>
#include <ostream>
#include <string>
#ifdef _WIN32
#include <stdio.h>
#endif
namespace raymii {
struct CommandResult {
std::string output;
int exitstatus;
friend std::ostream &operator<<(std::ostream &os, const CommandResult &result) {
os << "command exitstatus: " << result.exitstatus << " output: " << result.output;
return os;
}
bool operator==(const CommandResult &rhs) const {
return output == rhs.output &&
exitstatus == rhs.exitstatus;
}
bool operator!=(const CommandResult &rhs) const {
return !(rhs == *this);
}
};
class Command {
public:
/**
* Execute system command and get STDOUT result.
* Regular system() only gives back exit status, this gives back output as well.
* @param command system command to execute
* @return commandResult containing STDOUT (not stderr) output & exitstatus
* of command. Empty if command failed (or has no output). If you want stderr,
* use shell redirection (2&>1).
*/
static CommandResult exec(const std::string &command) {
int exitcode = 0;
std::array<char, 1048576> buffer {};
std::string result;
#ifdef _WIN32
#define popen _popen
#define pclose _pclose
#define WEXITSTATUS
#endif
FILE *pipe = popen(command.c_str(), "r");
if (pipe == nullptr) {
throw std::runtime_error("popen() failed!");
}
try {
std::size_t bytesread;
while ((bytesread = std::fread(buffer.data(), sizeof(buffer.at(0)), sizeof(buffer), pipe)) != 0) {
result += std::string(buffer.data(), bytesread);
}
} catch (...) {
pclose(pipe);
throw;
}
exitcode = WEXITSTATUS(pclose(pipe));
return CommandResult{result, exitcode};
}
};
}// namespace raymii
#endif//COMMAND_H