uasyncio on unix port

[kevinkk525]

Currently uasnycio is not compiled into the unix port but it does work if I copy the extmod/uasyncio directory into the library path of the unix port (and I think I had to change something in the build options but I can't remember, it's long ago and I'm currently setting up my PC again so can't test yet).

Would be good to have it available on the unix port too.

[jcw]

What I did was get a copy of the github repository, and add a symlink:

cd ~/.micropython/lib/
ln -s <path-to-repo>/extmod/uasyncio

This is on MacOS, but it should work fine on Linux as well.

[dpgeorge]

It might make sense to freeze uasyncio into the micropython-dev build. If using the standard micropython one can use the lib path as suggested by @jcw.

[kevinkk525]

What dev build?

To me it is strange that when I clone the repo and start building the firmware for different platforms, the esp32, esp8266 and pyboard all got the new uasyncio baked in but the unix port is still on V2 (or None iirc).
It's inconsistent and unexpected, creating additional work for a user to figure out how to install the new uasyncio that he is using on all other ports.

[jcw]

FWIW, duplicate discussion here: https://forum.micropython.org/viewtopic.php?f=15&p=48162

[kevinkk525]

That discussion was actually the reason why I opened this ticket.
Personally I always copied the uasyncio sources files into the lib directory, which was needed anyways because I was testing multiple versions of uasyncio so I had to change the files frequently.
Additionally I was used to that because I started using uasyncio before it was baked into the firmware by default, so I had to freeze it myself in other ports too.
But since it was baked into the firmware (and especially with the new uasyncio version), users don't have to worry about installing uasyncio manually anymore. Thus making the difference on the unix port more obvious.

[jcw]

I agree - it's always hard to commit to a feature freeze, but a fresh release would be very welcome.

Perfection is the enemy of done ...

[jimmo]

What dev build?

make VARIANT=dev

See #6131

[kevinkk525]

Thanks @jimmo
Since even the normal build is actually a "dev" build because it contains the latest commits to master, I personally don't see a point in even having a dev build but certainly not in having uasyncio only in the dev build.
Other ports have it in their standard build and to be consistent the unix port should work the same way.

[peterhinch]

@kevinkk525 I agree.

What's more, the dev build isn't working for me. On a fresh micropython clone on a Debian VM I built the dev variant with this result:

adminpete@debian8:~/micropython/ports/unix$ ./micropython-dev 
MicroPython v1.12-554-gce02d5e on 2020-06-18; linux version
Use Ctrl-D to exit, Ctrl-E for paste mode
>>> import uasyncio
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: no module named 'uasyncio'
>>> 

I also tried the coverage build, but that wouldn't compile:

adminpete@debian8:~/micropython/ports/unix$ make VARIANT=coverage
Use make V=1 or set BUILD_VERBOSE in your environment to increase build verbosity.
CC ../../py/mpstate.c
In file included from ../../py/mpstate.h:33:0,
                 from ../../py/mpstate.c:27:
../../py/misc.h:279:51: error: no previous declaration for ‘MP_COMPRESSED_ROM_TEXT’ [-Werror=missing-declarations]
 inline __attribute__((always_inline)) const char *MP_COMPRESSED_ROM_TEXT(const char *msg) {
                                                   ^
cc1: all warnings being treated as errors
../../py/mkrules.mk:63: recipe for target 'build-coverage/py/mpstate.o' failed
make: *** [build-coverage/py/mpstate.o] Error 1
adminpete@debian8:~/micropython/ports/unix$ 

[kevinkk525]

What's more, the dev build isn't working for me. On a fresh micropython clone on a Debian VM I built the dev variant with this result:

That's because #6131 isnt merged yet, so no uasyncio in dev build yet.

[peterhinch]

@kevinkk525 Thanks for that. Hopefully it will be in the standard build soon.

[metachris]

Please get uasyncio v3 into the standard build! 🙏

I've put together a pretty long micropython + esp introduction (that's almost ready). One of the things that's bugging me is the uasyncio situation... I'd like to recommend to readers to just stick with MicroPython 1.13+ and uasyncio v3. But I also want to encourage readers to write tests that can run against a local version of micropython. But the current local builds doesn't include uasyncio, and i think people would likely end up with the pycoopy one through upip.

In my opinion, getting uasyncio v3 into the standard build would go a long way on sorting out this whole uasyncio pain point!

[kevinkk525]

You may point anyone to building the dev build of the unix port. That one includes uasyncio v3. But I certainly agree, it's not good that the normal unix port is the only port that doesn't include the new uasyncio v3 by default.

[djantzen]

I was able to use uasyncio in the dev build (v1.18-169-g665f0e2a6), but now "const" is missing.

[robert-hh]

Should be there. But if not, you can always use the line below, if const is missing.

from micropython import const

or define const with a lambda function:

const = lambda x: x

Not as efficient as the real const, but at least the code works.

[djantzen]

True, but I have to modify library files with that directive in order for other 3rd party code to work. For example, os/__init__.py. I'm concerned there will be other surprises.

[n-elia]

Hi, I'll share how I'm currently building the standard Unix port with built-in uasyncio, if someone is interested.

The variants are defined within the directory ports/unix/variants. The document ports/unix/variants/manifest.py can be used to add frozen modules to the standard Unix port.

So, if you would like to add uasyncio, just append the following lines to ports/unix/variants/manifest.py:

freeze(
    "$(MPY_DIR)/extmod",
    (
        "uasyncio/__init__.py",
        "uasyncio/core.py",
        "uasyncio/event.py",
        "uasyncio/funcs.py",
        "uasyncio/lock.py",
        "uasyncio/stream.py",
        "uasyncio/task.py",
    ),
    opt=3,
)

and then recompile (make -B).

Alternatively, you can create your own subfolder under /variants, e.g. copying /standard, and adding a manifest.py containing the previous code. For instance, if you name the new variant as async, then you can build it through make VARIANT=async FROZEN_MANIFEST=variants/async/manifest.py -B.

I'm successfully using it with the Unix port in the master branch, version eb9674822b4d3be308beae2fa238ad85de9f09fa.
Hope this can help future visitors.

[LucasCTN]

Was PR #6131 enough to fix this? Is there something blocking uasyncio from being included in the unix port?

[jimmo]

Was PR #6131 enough to fix this? Is there something blocking uasyncio from being included in the unix port?

@LucasCTN yes, uasyncio is included on the dev variant of the unix port.

make VARIANT=dev

[LucasCTN]

@LucasCTN yes, uasyncio is included on the dev variant of the unix port.

make VARIANT=dev

Oh, that's great! Is there a plan to include in the default version? So it works out of the box?

[dpgeorge]

As of c1530a0, uasyncio is available in the standard unix build.