514 lines
17 KiB
Org Mode
514 lines
17 KiB
Org Mode
#+title: Today I Learned - lessons from building a C++ development stack
|
|
#+tags: @c++ @cmake @git @submodule @pybind11
|
|
#+description: lessons learned while building cooperating c++ libraries using nix|cmake|pybind11|eigen amongst others
|
|
#
|
|
# org-publish options
|
|
#
|
|
# ^:{} require a_{b} before assuming that b should be subscripted.
|
|
# without this option a_b will automatically subscript b.
|
|
#+options: ^:{}
|
|
#
|
|
# emacs-specific options
|
|
#+startup: showall
|
|
#
|
|
# html exporter options
|
|
#+language: en
|
|
#+keywords: c++ cmake git submodule pybind11 eigen
|
|
#+keywords: transitive-library-dependency
|
|
#+setupfile: ../ext/fniessen/theme-readtheorg.setup
|
|
# +infojs_opt: view:showall mouse:#ffc0c0 toc:nil ltoc:nil path:/web/ext/orginfo/org-info.js
|
|
# +html_head: <link rel="stylesheet" type="text/css" href="/web/css/primary.css" />
|
|
#+html_link_home: ../index.html
|
|
|
|
* wsl (windows services for linux)
|
|
|
|
- (Nov? 2023) [[file:../articles/2023/11/x11-apps-wedged-after-wsl-update.org][x11 apps wedged after wsl update]]
|
|
|
|
- (Sep 2024) [[file:../articles/2024/09/getting-fonts-from-nix.org][getting fonts from nix]]
|
|
|
|
* nix
|
|
|
|
- (Jun 2026) [[file:../articles/2026/06/nxfs-on-ubuntu.org][ptmx setup for nix on ubuntu]]
|
|
|
|
- (Jun 2026) [[file:../articles/2026/06/nix-binary-cache][nix binary cache]]
|
|
|
|
* git
|
|
|
|
- (May 2026) [[file:../articles/2026/05/git-subtree-bugfix.org][bugfix for git subtree]]
|
|
|
|
* webhosting + forgejo + CI
|
|
|
|
- (May 2026) [[file:../articles/2026/05/setup-forgejo-at-digital-ocean.org][forgejo setup]]
|
|
- (May 2026) [[file:../articles/2026/05/setup-forgejo-runner.org][forgejo runner setup)]]
|
|
|
|
- (Jun 2026) [[file:../articles/2026/06/setup-org2html-pipeline.org][self-hosted org-howto publishing]]
|
|
|
|
* cmake
|
|
|
|
- (Oct 2023) [[file:../articles/2023/10/inconsistent-eigen-package.org][inconsistent eigen package]]
|
|
|
|
- (Oct 2023) [[file:../articles/2023/10/cmake-headeronly-dependencies.org][cmake handling of header-only dependencies]]
|
|
|
|
** pybind11 link difficulties with transitive library dependencies (7oct2023)
|
|
|
|
*** Setup
|
|
|
|
- cmake version 3.25.3
|
|
- pybind11 version ???
|
|
- nix build (see https://github.com:rconybea/xo-nix2)
|
|
Consequences of nix build:
|
|
- Each package installed to a separate directory -- no "common swimming pool" like =/usr/lib=
|
|
- Implies install directory always distinct from any directory containing build inputs
|
|
- Tends to reveal oversights in toolchain, as we'll see below
|
|
|
|
- pybind library (=xo-pyreflect=) with dependency on a separate library (=xo-reflect=),
|
|
that in turn has secondary dependencies (=xo-refcnt=, =xo-indentlog=).
|
|
Note that =xo-indentlog= is header-only.
|
|
|
|
- Expect this cmake script to work:
|
|
#+begin_src cmake
|
|
find_package(pybind11)
|
|
pybind11_add_module(pyreflect pyreflect.cpp)
|
|
|
|
find_package(reflect CONFIG REQUIRED)
|
|
target_link_libraries(pyreflect PUBLIC reflect)
|
|
#+end_src
|
|
|
|
*** Problem
|
|
|
|
- Instead, link fails. Link line something like:
|
|
#+begin_example
|
|
g++ -fPIC ... -o pyreflect.cpython-311-x86_64-linux-gnu.so /path/to/libreflect.so -lrefcnt -lindentlog
|
|
#+end_example
|
|
|
|
Two problems here:
|
|
1. directory containing =librefcnt.so= isn't on the link line (no =-L/path/to/refcnt/dir= for example).
|
|
2. =libindentlog.so= does not exist, since =indentlog= is header-only
|
|
|
|
- Looked into intermediate outputs like =lib/cmake/reflectTargets.cmake=, excerpt:
|
|
#+begin_src cmake
|
|
set_target_properties(reflect PROPERTIES
|
|
INTERFACE_INCLUDE_DIRECTORIES "${_IMPORT_PREFIX}/include"
|
|
INTERFACE_LINK_LIBRARIES "indentlog;refcnt"
|
|
)
|
|
#+end_src
|
|
It's not obvious how =xo_pyreflect= can know that =indentlog= is header-only, while =refcnt= isn't
|
|
(though could presumably extract the relevant libdir from =find_package()= with some work).
|
|
|
|
*** Workaround
|
|
|
|
- Recognize that =pyreflect= link shouldn't need =refcnt= on the link line,
|
|
since =libreflect.so= has a =DT_NEEDED= entry for it.
|
|
|
|
#+begin_example
|
|
$ readelf -d /path/to/libreflect.so
|
|
|
|
Dynamic section at offset 0x17860 contains 34 entries:
|
|
Tag Type Name/Value
|
|
0x0000000000000001 (NEEDED) Shared library: [librefcnt.so.1]
|
|
...
|
|
#+end_example
|
|
|
|
- When building =pyreflect=, suppress transitive dependencies
|
|
For example:
|
|
|
|
#+begin_src cmake
|
|
# xo_cxx.cmake
|
|
macro(xo_pybind11_dependency target dep)
|
|
find_package(${dep} CONFIG REQUIRED)
|
|
set_property(TARGET ${dep} PROPERTY INTERFACE_LINK_LIBRARIES "")
|
|
target_link_libraries(${target} PUBLIC ${dep})
|
|
endmacro()
|
|
#+end_src
|
|
|
|
Then in =.cmake= for =pyreflect=, something equivalent to:
|
|
|
|
#+begin_src cmake
|
|
pybind11_add_module(pyreflect pyreflect.cpp)
|
|
xo_pybind11_dependency(pyreflect reflect)
|
|
#+end_src
|
|
|
|
* lsp (language server process)
|
|
|
|
** mysterious lsp complaints about iostream headers (22feb2024)
|
|
|
|
*** Setup
|
|
|
|
- working on bespoke streambuf implementation (cmake-examples/zstream/include/zstream/zstreambuf.hpp)
|
|
|
|
*** Problem
|
|
|
|
- getting mysterious errors from flymake + emacs, e.g. message "<fstream> not found"
|
|
|
|
*** Solution
|
|
|
|
- Had discarded =cmake-examples/build= directory, using =cmake-examples/.build= instead.
|
|
This was to prevent build tree showing up when running =tree= in the =cmake-examples= directory
|
|
|
|
- Left =cmake-examples/compile_commands.json= as a broken symlink referring to old build directory
|
|
|
|
- This causes =lsp= understanding of compiler invocation to deteriorate as project acquires new files
|
|
|
|
- Fix by swinging symlink to =cmake-examples/.build/compile_commands.json=, duh!
|
|
|
|
* iostream
|
|
|
|
** general api rant (25feb2024)
|
|
|
|
*** =istream.read()= doesn't report the number of bytes/chars read.
|
|
Instead of:
|
|
#+begin_src c++
|
|
istream & istream::read (char_type * s, std::streamsize count);
|
|
#+end_src
|
|
|
|
I'd prefer signature
|
|
#+begin_src c++
|
|
istream & istream::read (char_type * s, std::streamsize count, std::streamsize * p_gcount);
|
|
#+end_src
|
|
|
|
Developers are expected to use.
|
|
#+begin_src c++
|
|
std::streamsize istream::gcount () const;
|
|
#+end_src
|
|
|
|
I think this is inferior, since relies on state held by =istream=,
|
|
that will be discarded on next read operation.
|
|
|
|
*** =istream.read(s, n)= expects always to read n chars.
|
|
It sets =failbit= if less than =n= chars read.
|
|
|
|
Apparent alternatives are unsatisfactory:
|
|
|
|
1. =istream & readsome(s, n)= isn't required to do any physical i/o; instead reports what's available already in memory
|
|
2. =istream & get(s, n, delim)= only reads up to first occurence of =delim=.
|
|
3. =istream & get(s, n)= is just a convenience for =istream::get(s, n, '\n')=.
|
|
4. could try writing a loop using combination of =istream::sync()=, =istream::readsome()=, but that won't work if istream is actually unbuffered.
|
|
5. =istream s; s.rdbuf()->sgetn(s, n)= bypasses =istream= code for sentry object etc, and can't set istream's =eofbit=.
|
|
|
|
The following workaround is viable, except that it will read one-byte-at-a-time if input alternates between bytes values ='\x0'= and ='\xff'=:
|
|
|
|
#+begin_src c++
|
|
template<typename istream>
|
|
std::streamsize
|
|
read_upto(istream & in, istream::char_type * s, std::streamsize n)
|
|
{
|
|
std::streamsize n_read = 0;
|
|
|
|
constexpr char c_bits = '\x0'; /*any char value will do here*/
|
|
|
|
char delim = c_bits;
|
|
|
|
for (; in.good() && !in.eof() && (n_read < n); delim = delim ^ '\xff') {
|
|
// each iteration alternates between {c_bits, ~c_bits} as delimiter,
|
|
// so guarantees at least one byte progress every two iterations
|
|
|
|
in.get(s, n, delim);
|
|
|
|
std::streamsize nr = in.gcount();
|
|
if (nr > 0) {
|
|
n_read += nr;
|
|
s += nr;
|
|
}
|
|
}
|
|
|
|
return n_read;
|
|
}
|
|
#+end_src
|
|
|
|
I'd prefer to support this behavior (without the performance-accident-waiting-to-happen) directly from =istream=.
|
|
|
|
Another strategy is to use =istream::peek()= to check for input and =istream::readsome()= to fetch it
|
|
|
|
#+begin_src c++
|
|
template<typename istream>
|
|
std::streamsize
|
|
read_upto(istream & in, istream::char_type * s, std::streamsize n)
|
|
{
|
|
std::streamsize n_read = 0;
|
|
|
|
while (in.good() && !in.eof() && (n_read < n))) {
|
|
in.peek(); /* ensure at least one byte available in streambuf */
|
|
|
|
std::streamsize nr = in.readsome(s + n_read, n - n_read);
|
|
|
|
n_read += nr;
|
|
}
|
|
}
|
|
#+end_src
|
|
|
|
This works if =streambuf= actually does buffering. It may be very slow if =streambuf= is unbuffered.
|
|
|
|
=istream::sentry= looks interesting, but doesn't do any reading (except to possibly skip whitespace).
|
|
|
|
gcc 12.2.0's implementation:
|
|
|
|
#+begin_src c++
|
|
template<typename _CharT, typename _Traits>
|
|
basic_istream<_CharT, _Traits>::sentry::
|
|
sentry(basic_istream<_CharT, _Traits>& __in, bool __noskip) : _M_ok(false)
|
|
{
|
|
ios_base::iostate __err = ios_base::goodbit;
|
|
if (__in.good())
|
|
{
|
|
__try
|
|
{
|
|
if (__in.tie())
|
|
__in.tie()->flush();
|
|
if (!__noskip && bool(__in.flags() & ios_base::skipws))
|
|
{
|
|
const __int_type __eof = traits_type::eof();
|
|
__streambuf_type* __sb = __in.rdbuf();
|
|
__int_type __c = __sb->sgetc();
|
|
|
|
const __ctype_type& __ct = __check_facet(__in._M_ctype);
|
|
while (!traits_type::eq_int_type(__c, __eof)
|
|
&& __ct.is(ctype_base::space,
|
|
traits_type::to_char_type(__c)))
|
|
__c = __sb->snextc();
|
|
|
|
// _GLIBCXX_RESOLVE_LIB_DEFECTS
|
|
// 195. Should basic_istream::sentry's constructor ever
|
|
// set eofbit?
|
|
if (traits_type::eq_int_type(__c, __eof))
|
|
__err |= ios_base::eofbit; // (A)
|
|
}
|
|
}
|
|
__catch(__cxxabiv1::__forced_unwind&)
|
|
{
|
|
__in._M_setstate(ios_base::badbit);
|
|
__throw_exception_again;
|
|
}
|
|
__catch(...)
|
|
{ __in._M_setstate(ios_base::badbit); }
|
|
}
|
|
|
|
if (__in.good() && __err == ios_base::goodbit) // (B)
|
|
_M_ok = true;
|
|
else
|
|
{
|
|
__err |= ios_base::failbit; // (C)
|
|
__in.setstate(__err);
|
|
}
|
|
}
|
|
#+end_src
|
|
|
|
with
|
|
#+begin_src c++
|
|
template<typename _Facet>
|
|
inline const _Facet&
|
|
__check_facet(const _Facet* __f)
|
|
{
|
|
if (!__f)
|
|
__throw_bad_cast();
|
|
return *__f;
|
|
}
|
|
#+end_src
|
|
|
|
Note that if =__noskipws= is =false= and sentry encounters eof,
|
|
then the line marked (A) executes --> test (B) fails --> (C) executes,
|
|
flagging stream as in an 'unrecoverable error state'.
|
|
The line (A) appears to be mandatory (in spite of the inline comment).
|
|
|
|
From https://cppreference.com:
|
|
|
|
#+begin_quote
|
|
explicit sentry( std::basic_istream<CharT, Traits>& is, bool noskipws = false );
|
|
|
|
Prepares the stream for formatted input.
|
|
|
|
If is.good() is false, calls is.setstate(std::ios_base::failbit) and returns.
|
|
Otherwise, if is.tie() is not a null pointer, calls is.tie()->flush() to synchronize the output sequence with external streams.
|
|
This call can be suppressed if the put area of is.tie() is empty.
|
|
The implementation may defer the call to flush() until a call of is.rdbuf()->underflow() occurs.
|
|
If no such call occurs before the sentry object is destroyed, it may be eliminated entirely.
|
|
|
|
If noskipws is zero and is.flags() & std::ios_base::skipws is nonzero,
|
|
the function extracts and discards all whitespace characters until the next available character is not a whitespace character
|
|
(as determined by the currently imbued locale in is).
|
|
If is.rdbuf()->sbumpc() or is.rdbuf()->sgetc() returns traits::eof(),
|
|
the function calls setstate(std::ios_base::failbit | std::ios_base::eofbit)
|
|
(which may throw std::ios_base::failure).
|
|
|
|
Additional implementation-defined preparation may take place,
|
|
which may call setstate(std::ios_base::failbit) (which may throw std::ios_base::failure).
|
|
|
|
If after preparation is completed, is.good() == true, then any subsequent calls to operator bool will return true.
|
|
#+end_quote
|
|
|
|
However we can bypass this with =__noskip_= set to =true=:
|
|
|
|
#+begin_src c++
|
|
template<typename istream>
|
|
std::streamsize
|
|
read_upto(istream & in, istream::char_type * s, std::streamsize n)
|
|
{
|
|
istream::sentry sentry(in, true /*noskipws*/);
|
|
|
|
std::streamsize n_read = 0;
|
|
|
|
if (sentry) {
|
|
try {
|
|
n_read = in.rdbuf()->sgetn(s, n);
|
|
|
|
in.setstate(ios::eofbit);
|
|
} catch(__cxxabiv1::__forced_unwind &) {
|
|
in.setstate(ios::failbit);
|
|
throw;
|
|
} catch(...) {
|
|
in.setstate(ios::failbit);
|
|
}
|
|
}
|
|
|
|
return n_read;
|
|
}
|
|
#+end_src
|
|
|
|
Another alternative would be to post-process =read()=, and clear =failbit= if set along with =eofbit=:
|
|
|
|
#+begin_src c++
|
|
template<typename istream>
|
|
std::streamsize
|
|
read_upto(istream & in, istream::char_type * s, std::streamsize n)
|
|
{
|
|
in.read(s, n);
|
|
|
|
std::streamsize n_read = in.gcount();
|
|
|
|
if ((n_read < n) && in.eof() && in.fail()) {
|
|
/* clear failbit */
|
|
in.clear(in.rdstate() & ~std::ios::failbit);
|
|
}
|
|
|
|
return n_read;
|
|
}
|
|
#+end_src
|
|
|
|
*** Iostream get isn't monotonic
|
|
|
|
=iostream.get(s, n, delim)= sets =failbit= if first character matches delim.
|
|
|
|
This interferes with using =iostream.get= as building block for a longer i/o sequence;
|
|
|
|
Tripped over this while writing =zstream.read_until= for my =cmake-examples= project:
|
|
|
|
Instead of:
|
|
|
|
#+begin_src c++
|
|
std::streamsize read_until(char_type * s,
|
|
std::streamsize n,
|
|
bool check_delim_flag,
|
|
char_type delim)
|
|
{
|
|
...
|
|
|
|
std::streamsize nr = 0;
|
|
|
|
this->get(s, n, delim);
|
|
nr = this->gcount();
|
|
|
|
...
|
|
|
|
return nr;
|
|
}
|
|
#+end_src
|
|
|
|
We need carve-out:
|
|
|
|
#+begin_src c++
|
|
std::streamsize read_until(char_type * s,
|
|
std::streamsize n,
|
|
bool check_delim_flag,
|
|
char_type delim)
|
|
{
|
|
...
|
|
|
|
std::streamsize nr = 0;
|
|
|
|
int_type nextc = this->rdbuf_.sgetc();
|
|
|
|
if (nextc == Traits::to_int_type(delim)) {
|
|
nr = 0;
|
|
} else {
|
|
this->get(s, n, delim);
|
|
|
|
nr = this->gcount();
|
|
}
|
|
|
|
...
|
|
|
|
return nr;
|
|
}
|
|
#+end_src
|
|
|
|
|
|
|
|
*** Iostream position reporting isn't monotonic.
|
|
|
|
=iostream.tellg()= and =iostream.putg()= report current position w.r.t.
|
|
beginning of stream for input (get) and output (put) respectively.
|
|
|
|
Unfortunately, they are not monotonic, and code like this is subtly broken:
|
|
|
|
#+begin_src c++
|
|
istream & input = ...; // some binary stream
|
|
struct foo part1;
|
|
struct foo part2;
|
|
|
|
istream::pos_type p0 = input.tellg();
|
|
|
|
input >> part1 >> part2;
|
|
|
|
istream::pos_type p1 = input.tellg();
|
|
|
|
istream::pos_type n_read = p1 - p0;
|
|
#+end_src
|
|
|
|
If stream reaches end-of-file at the end of =part2=, then in fact reading was successful,
|
|
but =p1= will be =-1=, and =n_read= will be nonsense.
|
|
|
|
Presumably this is why =iostream.gcount()= exists: otherwise there'd be no way to
|
|
determine how many bytes/chars a preceding read obtained.
|
|
|
|
A correct (but awkward and error-prone) implementation:
|
|
|
|
#+begin_src c++
|
|
istream & input = ...;
|
|
struct foo part1;
|
|
struct foo part2;
|
|
|
|
std::streamsize n_read = 0;
|
|
|
|
input >> part1;
|
|
n_read += input.gcount();
|
|
|
|
input >> part2;
|
|
n_read += input.gcount();
|
|
#+end_src
|
|
|
|
*** Streambuf not responsible for eofbit.
|
|
|
|
=istream.eofbit= probably belongs in =streambuf=.
|
|
=streambuf= has to recognize end-of-file anyway, since it's responsible for physical I/O.
|
|
It might as well record and report it.
|
|
|
|
*** Stream position reporting from streambuf
|
|
|
|
It would be simpler for =streambuf= to support =istream::tellg()= and =istream::tellp()= directly instead of relying on =streambuf::seekoff()=.
|
|
Argument here is that even for a non-seekable stream buffer, it still makes sense to support =tellg()= and =tellp()=.
|
|
This requires streambuf author to implement at least a restricted version of =streambuf::seekoff()=,
|
|
which muddies the waters.
|
|
|
|
(ed: switching to article-based format)
|
|
|
|
* zlib
|
|
|
|
- (Jan? 2024) [[file:../articles/2024/01/zstream-isnt-moveable.org][zlib z_stream cannot be moved]]
|
|
|
|
* gcc
|
|
|
|
- (June 2024) [[file:../articles/2024/06/gcc-builtin-defines.org][get builtin preprocessor defines from gcc]]
|
|
|
|
* llvm
|
|
|
|
- (June 2024) [[file:../articles/2024/06/injecting-symbols-into-llvm-jit.org][injecting symbols into llvm jit]]
|
|
- (June 2024) [[file:../articles/2024/06/llvm-cpp-function-types.org][alloca and function pointers]]
|