[ldoolitt]

Add brain-dead support for OpenBSD in the tangle of ifdefs surrounding memalign().

Better would be for an SME to simplify the ifdefs, not add a new one.
But I'm not that SME.

Without this patch, attempting a pip install of cothreads on OpenBSD gives a build error:

context/cocore.c:37:10: fatal error: 'malloc.h' file not found

With this fix, pip install works, and the result is good enough to run cocotb.

[Araneidae]

Thank you, this looks like a very straightforward fix. OpenBSD is quite a blast from the past for me (though I spent more time with FreeBSD)! I guess there are similar issues with FreeBSD, but I imagine you've not tried that?

[Araneidae]

We don't need nested #if here, instead just note that __APPLE__ and __OpenBSD__ are equivalent in this context. Write this instead as

#if !(defined(__APPLE__) || defined(__OpenBSD__))

or maybe (more direct but less clear)

#if !defined(__APPLE__) && !defined(__OpenBSD__)

[ldoolitt]

@Araneidae thanks for your response!
Yes, your suggestion emphasizes the equivalence of APPLE and OpenBSD, and uses fewer (but slightly more intricate) lines of code. Fine by me.
I pushed such changes to my fork, and confirmed everything still Works For Me.

I haven't tried FreeBSD. It's very likely to work the same as OpenBSD, as you say, but I am reluctant to add that feature without testing it. And no, I don't currently have any *BSD machines available to me besides OpenBSD 8.0.

After probing a bit more, I see my standards for "Works For Me" are pretty low. It has to install to keep pip happy installing cocotb. I have my doubts that my tests involving cocotb actually pull in cothread. If I simply import cothread in python, it segfaults.

[python3.14]21107/225416 sp=4589d182ed8 inside 6fefd24ce000-6fefd28cdfff: not MAP_STACK

I don't know if I'm qualified to debug this. Any hints?

[Araneidae]

Thank you, that looks very straightforward.

I was just about to merge, but then our CI seems to be busted at the moment ... and I see you now have a deeper problem.

I can make an educated guess. Cothread allocates blocks of memory for fresh areas of stack, and the "not MAP_STACK" message suggest to me that OpenBSD is (somehow) checking that the stack pointer is pointing to a block of memory without this magic MAP_STACK flag (this appears to be an option to mmap). I know that OpenBSD is very security oriented, but I didn't know about this flag. It looks like OpenBSD checks the stack pointer on system calls, and that's probably what's biting us here.

This is fixable, I think, but it's not easy. We'd need to rework

cothread/context/cocore.c

Lines 239 to 271 in 0eaef07

	static struct stack *create_stack(
	struct cocore *coroutine, size_t stack_size, bool check_stack,
	int guard_pages)
	{
	/* Align the stack either by its minimum alignment or to pages if guard
	* pages have been requested. */
	size_t guard_size = guard_pages * page_size;
	size_t alignment = guard_size == 0 ? STACK_ALIGNMENT : page_size;
	stack_size = (stack_size + alignment - 1) & -alignment;
	size_t alloc_size = stack_size + guard_size;
	void *alloc_base = MALLOC_ALIGNED(alignment, alloc_size);
	if (guard_size > 0)
	/* Disable access to the guard area. */
	mprotect(
	FRAME_START(alloc_base + stack_size, alloc_base),
	guard_size, PROT_NONE);
	struct stack *stack = malloc(sizeof(struct stack));
	stack->stack_base = STACK_BASE(alloc_base, stack_size + guard_size);
	stack->stack_size = stack_size;
	stack->guard_size = guard_size;
	stack->valgrind_stack_id =
	VALGRIND_STACK_REGISTER(alloc_base, alloc_base + alloc_size);
	stack->check_stack = check_stack;
	stack->current = coroutine;
	stack->ref_count = 1;
	if (check_stack)
	memset(FRAME_START(alloc_base, alloc_base + guard_size),
	0xC5, stack_size);
	return stack;
	}

so that the allocation

cothread/context/cocore.c

Line 250 in 0eaef07

void *alloc_base = MALLOC_ALIGNED(alignment, alloc_size);

returns a MAP_STACK capable block of memory.

It's possible that OpenBSD already has a mechanism for this, or we'd have to use mmap. Doable ... but am not sure it's worth it, and I'm not really in a position to investigate.

I think the PR is still valid, and can be a placeholder for further *BSD support.

[AlexanderWells-diamond]

I've just fixed the CI in PR #88. Please rebase your branch to pick up the changes.

[ldoolitt]

I merged the latest "upstream" changes (thanks, @AlexanderWells-diamond !), used the mmap() suggestion from @Araneidae, and pushed again.
The result is installable, and can at least get imported and print its version.
Not surprisingly, cocotb in this environment also still works -- although that doesn't seem to mean much.

Are there any obvious self-tests I should run, as well? I don't (yet) have any epics installed.

[Araneidae]

I wrote a comment against the latest commit 9e58d11#commitcomment-207892143. I wonder if I should copy it into this thread?

Hmm. We're going to need to improve the testing here! I wonder if it would be possible to run the https://github.com/DiamondLightSource/pythonSoftIOC test bench on OpenBSD? @AlexanderWells-diamond, do you think you'll be able to help @ldoolitt with this? I suspect it might be a slightly miserable experience, but it's probably worth a try.

[Araneidae]

See 9e58d11#commitcomment-207892143!

[Araneidae]

Actually, here's the short form. Try:

#define FREE_ALIGNED(mem, length)  munmap(mem, length)

and make all the associated required changes (including adding length to all the other #defines and fixing the call), and see if this works.

This is more expensive than malloc/free ... but then we're doing system calls for the guard pages anyway, so this is probably ok.

[AlexanderWells-diamond]

To run the cothread tests run these commands:

pip install -e .[dev]
tox -e pytest

I'm not sure exactly what will happen if you don't have EPICS on your PATH. The tests will likely have some failures. Let us know the results and we can take a look.

We have a more extensive testbench available in the pythonSoftIOC repo. If you are willing, downloading and installing that will likely provide a better set of tests. To try that do:

git clone https://github.com/DiamondLightSource/pythonSoftIOC.git
cd pythonSoftIOC/
git submodule init 
git submodule update
python -m venv venv
source venv/bin/activate
pip install -e .[dev]
pytest

That version should include epicscorelibs, which is EPICS with Python bindings.

[ldoolitt]

pip install -e .[dev]

My first attempt, at least, bombed out trying to build the pipdeptree (4.2.13) dependency.

      ../meson.build:29:8: ERROR: Program 'cargo' not found or not executable

git clone https://github.com/DiamondLightSource/pythonSoftIOC.git
[...]
pip install -e .[dev]

That version should include epicscorelibs, which is EPICS with Python bindings.

OpenBSD is not on epicscorelibs list of supported platforms, so it doesn't build/install.
Pretty lame list of platforms, if you ask me. In src/python/epicscorelibs/config.py:

# map from python system name to epics OS_CLASS
    osname = conf['OS_CLASS'] = {
        'Linux': 'Linux',
        'Windows': 'WIN32',
        'Darwin': 'Darwin',
    }[platform.system()]

I'm fond of drawing an analogy between the modern state of software dependencies and
a Sierpiński curve. It's easy to slip into the case where everything depends on everything else,
even if many of the dependencies are weak, like the points where triangles touch.
It would be great to cut some of those dependencies every once in a while!

In that spirit, I may be able to disconnect my immediate task from cothread entirely!
But EPICS does interest me, so I don't mind continuing a bit more here.

[Araneidae]

@ldoolitt, I'm full of sympathy with your feelings about modern dependency webs.

Before tangling with Python integration I'd suggest just trying to compile EPICS base out of the box; I mean, just download EPICS base and try to build it. I'm going to guess that that will be full of enough surprises on OpenBSD to keep you entertained. Before EPICS v7 I could download EPICS base, change into the root directory, type make and it would just build ... I doubt you'll have the same experience though... Best of luck :)