RobotLab 2021: JupyterLab 3, Robot Framework 4

[bollwyvl]

Let's get the band back together!

There have been so many nice things published recently that it's worth warming this back up.

Some ideas (please feel free to add more):

Packages

new?	package	version	conda-forge?	rf4?	notes
0	Python	3.8	1	1
0	JupyterLab 3	3.0.x	1	-
0	Robot Framwork	4.x	1	1
0	Firefox	78 LTS	1	-
?	xeus-robot	0.3.3	1	1	discussion below
0	robotkernel	1.5.0	1	1
0	robotframework-jupyterlibrary	0.3.1	1	1
1	jupyterlab_robotmode	0.3.1	1	1
0	jupyterlab-starters	1.0.2	1	-
1	jupyterlab-tour	3.0.1	1	-
1	jupyterlab-lsp	3.6.0	1	-
1	robotframework-lsp	0.15.0	1	1
1	robotframework-robocop	1.7.1	1	1
0	restinstance	1.0.2	1	?

Infra

• adapt build from gt-coarl-lab
  • use doit
  • use GitHub Actions (it actually generates most of the workflow scripts)

for posterity:

Hooray JupyterLab 2.0!

https://pypi.org/project/jupyterlab/2.0.0/

Looks like RF 3.2 is also dangerously close:

https://github.com/robotframework/robotframework/milestone/48

It may also be worth considering moving to python 3.8, as the whole Jupyter stack (at least) is good to go on windows (that was a mess 😝)

I can't commit to a bunch of time to move the ball forward right now, but am going to start tinkering on various things (like jupyterlab-starters) so that we're in a better spot when it's go-time.

[bollwyvl]

started on starters: deathbeds/jupyterlab-starters#39

[datakurre]

I’ll probably try to fix RF 3.2b2 compat on Tuesday. It allows robotkernel to work fully on public API.

Update: Fell into rabbit holes. Continuing later.

[bollwyvl]

🎉 RF 3.2! Guess it's time to take a proper look at actually doing a new release.

I see robotkernel 1.4.0 is already out (congrats!), but some things we may want to address first:

• Investigate image library #65
• Migrate to python 3.8? #68
• Add RF 3.2 parser features to syntax highlighting robotkernel#47
• Support JupyterLab 2.0 robotframework-jupyterlibrary#20
• add robotkernel 1.4.0 conda-forge/staged-recipes#11394

[datakurre]

#65 could be skipped for this.

[datakurre]

Btw, what is the state of JupyterLab LSP?

[bollwyvl]

jlsp is working pretty well, and supports lab 2.x: there's some clutch work going on to support dynamic configuration by the client, which certain language servers pretty much require for some of their cooler features. i haven't checked in on rflsp in a while...

[bollwyvl]

ha, and 3.2 broke our pipeline on jlsp. funny.

[bollwyvl]

Here's a little thing that puts together both jupyterlab-lsp and jupyterlab-debugger:

https://github.com/blink1073/jupyter-ide-demo

It would be super rad, of course, to be able to add breakpoint debugging to robotkernel, but the plumbing under the current debugger is based on https://github.com/jupyter-xeus/xeus-python, not ipykernel 😿 so would be a non-trivial amount of work.

[datakurre]

Cool! I have assumed that break point debugging would require turning kernel async and implementing custom robot runner 🤔

…

On 30. Apr 2020, 1.43 +0300, Nicholas Bollweg ***@***.***>, wrote: Here's a little thing that puts together both jupyterlab-lsp and jupyterlab-debugger: https://github.com/blink1073/jupyter-ide-demo It would be super rad, of course, to be able to add breakpoint debugging to robotkernel, but the plumbing under the current debugger is based on https://github.com/jupyter-xeus/xeus-python, not ipykernel 😿 so would be a non-trivial amount of work. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[bollwyvl]

Hey hey! Long time no talk!

Some updates:
xeus-robot has been released, and supports the debug adapter protocol, and works with the jupyterlab debugger! I ❤️ irobotframework and robotkernel, but seems we should probably consider a build that includes this new kernel (and robotkernel).

JupyterLab 3 (in RC) has been (partially) internationalized, and most importantly to me, doesn't need end-user nodejs to add extensions. I'm currently trying to smooth things out with the weirdness that is codemirror's extend-by-import approach with Lab 3 by just shipping the simple mode addon, and hopefully it makes the cut. Once that's through, i think maybe we make a new jupyterlab_robotmode repo and start tinkering there (#47).

These sound like some very interesting things coming together!

[datakurre]

So great to hear from you!

All these changes have proven that RobotLab as a stablish and out of the box working release was a great thing.

Sounds like a lot of work ahead. I hope to get more close to robot stuff again later this year. The next RoboCon will be online event in January, but probably still too early for me to fix everything. Another big change is that RobotFramework community has now a playwright based alternative (robotframework-browser) for Selenium for browser testing and I have not been able to look into that yet. Also Whit me has some replacement for Windows testing / automation so all my “magic” needs rewelriting in the future 🙈

I’ve also been following RoboCorp development. As I expected, they had to heavily fork the kernel to match their goals. But it also looks like their product is so tied to their platform that there remains room for RobotLab.

[bollwyvl]

Sounds like a lot of work ahead.

Yeah, it never ends! Nearest term, are you good with me starting a third repo for the syntax stuff for lab 3, targeting a pypi release? This would also let me warm robotframework-jupyterlibrary back up before trying to do the whole enchilada on this repo.

Over here, we've already done pretty well to start reducing the complexity of doing these releases by upstreaming things, and Lab 3 will bring that down even further, as we won't have to rebuild lab with the custom extensions. I've been experimenting with some different approaches (doit, conda-lock) to make some of these steps more reproducible... it still ain't nix, but what is? 🤣

playwright based alternative

ha, and I keep trying to get nodejs out of my projects!

so all my “magic” needs rewelriting in the future

these are pretty much the coolest things ever, though. good luck trying to make it work with everything!

there remains room for RobotLab.

absolutely! even if we aren't putting out releases all the time, this work has been instrumental in demonstrating some key overall system concepts. It remains one of my favorite testing examples.

[datakurre]

@bollwyvl Yes, it is completely ok to create a new repo for the syntax stuff. The current package inside robotkernel is already almost 1:1 copy of ipythonrobotframework's version.

[datakurre]

I am curious, how did JupyterLab solve the re-compilation after every extension issue? A link to somewhere is enough :-)

[bollwyvl]

It took a change to webpack core. Here's the magic sauce: https://webpack.js.org/concepts/module-federation/ And where the bulk of the work happened: jupyterlab/jupyterlab#8385 Basically, an extension needs a devDependency on @jupyterlab/builder, the dev runs "labextension build" to generate an opaque remoteEntry.js file pointing at all of the bundles it might need, and then makes sure those all those files end up in data_files for a pip package. At runtime, it grabs all of the remoteEntry files, served from a composite static_path like how nbextensions work, and as each dependency is loaded, it does a real semver solve... So it's still possible to get duplicates or unpredictable behavior... as we've seen, codemirror can still create some issues, as it relies on import side-effects. but for the most part will Just Work.