Autotune enabled in GQC. #9904

batinkov · 2021-10-07T11:03:51Z

A new UI component with the corresponding model are added. They enables Autotune in GQC. The autotuning can be performed only when the vehicle is flying. The new Autotune UI and the old interface are put under a switch and only one of them is available to the pilot at at time.

Autotune.QGC.mp4

batinkov · 2021-10-07T11:08:06Z

@bkueng and @MatejFranceskin could you have a look? Some things from the model are still missing and I'm working on them. The reason I drafted a PR is because the change is big and I'm guessing there will be comments to address.

bkueng · 2021-10-19T07:48:05Z

On the UI side I see 2 things we need to make sure:

Not all controllers support auto-tuning
The current auto-tuning for MC applies to both rate controller and attitude controller

Not sure how to best reflect that, maybe only show the button for the rate controller tab and mention that it applies to both rate + attitude?

bresch · 2021-10-19T12:23:21Z

src/Vehicle/Autotune.cpp

+    stopTimers();
+    _autotuneInProgress = false;
+    _disarmMessageDisplayed = false;
+    _autotuneStatus = tr("Autotune: Vehicle failed");


Suggested change

_autotuneStatus = tr("Autotune: Vehicle failed");

_autotuneStatus = tr("Autotune: Failed");

Already fixed.

bresch · 2021-10-19T12:32:53Z

Not all controllers support auto-tuning
The current auto-tuning for MC applies to both rate controller and attitude controller
Not sure how to best reflect that, maybe only show the button for the rate controller tab and mention that it applies to both rate + attitude?

Yes, same for fixedwings, only the rate and attitude controllers are tuned (not velocity and position), so the autotune button should only appear on the rate and attitude loop tabs. We could add the following message above the button: "By clicking on the "Autotune" button below, the drone will automatically tune its angular rate and attitude controllers".

…review.

batinkov · 2021-10-22T10:05:44Z

@bkueng
You're right not all controllers support Autotune but there is no way to know this in advance. If a controller does not support it the operation will fail. The manual tuning will always be there.

@bresch
You're comment above is addressed

@bkueng and @bresch
I tweaked the Autotune and now it appears only in rate and attitude loop tabs as requested. Have a look at the updated video.

bresch · 2021-10-22T10:27:47Z

@batinkov Thanks. We might want to show something different in the plot at some point (e.g.: currently computed PID parameters, convergence of the algorithm, ...) but it requires more changes (e.g.: passing the data through mavlink) so it's good like that for now.

bkueng · 2021-10-25T05:44:37Z

src/QmlControls/CMakeLists.txt

@@ -50,6 +50,7 @@ add_custom_target(QmlControlsQml
 		EditPositionDialog.qml
 		ExclusiveGroupItem.qml
 		FactSliderPanel.qml
+        AutotuneUI.qml


@bkueng fixed :)

DonLakeFlyer

Sorry for taking so long.

DonLakeFlyer · 2021-10-25T17:06:24Z

src/Vehicle/Autotune.cpp

+    if (autotune->_autotuneInProgress) {
+        if ((commandResult == MAV_RESULT_IN_PROGRESS) || (commandResult == MAV_RESULT_ACCEPTED)) {
+            autotune->handleAckStatus(progress);
+        }


Coding style of if/else

if () { } else { }

DonLakeFlyer · 2021-10-25T17:07:59Z

src/Vehicle/Autotune.cpp

+    stopTimers();
+    _autotuneInProgress = false;
+    _disarmMessageDisplayed = false;
+    _autotuneStatus = tr("Autotune: Failed");


I think there should be better handling of autotune being unsupported by the firmware. As far as I can tell it looks like all the user will get is a "failed" status which is not super clear. The ack will tell if the command is unsupported. In that case the user should get a clear message that Autotune is not supported by this vehicle.

@bkueng @DonLakeFlyer Can a frame know it supports the feature in advance?

In that case maybe you could send MAV_CMD_DO_AUTOTUNE_ENABLE.param1 with value 0 (no operation) as a test - the vehicle could ack with MAV_RESULT_UNKNOWN to indicate it doesn't understand the command for the current frame type.

It might of course be better to explicitly define that param1=0 is a no-op test.

If not, we could have a protocol bit, or component information.

I was thinking to add it to a new comp info file, but your other options would work here too.
There's more small bits like this, for example if sensor calibration reset to factory is supported. Comp info would make this simple to extend and we can add as many as we want.

DonLakeFlyer · 2021-10-25T17:12:12Z

src/Vehicle/Vehicle.cc

@@ -386,6 +387,8 @@ void Vehicle::_commonInit()

    _objectAvoidance = new VehicleObjectAvoidance(this, this);

+    _autotune = _firmwarePlugin->createAutotune(this);


Does the autotune object really need to be in existence all of the time? Seems like this is only needed when autotune is running. Somewhere in the Tuning Qml hierarchy there is a controller isn't there. Couldn't the autotune C++ code be in there? That way the autoune code is only in existence when the Autotune UI is up.

files added in mavlink#9904 - AutotuneUI.qml was added twice - Autotune.cpp was not added to CMakeLists.txt

files added in #9904 - AutotuneUI.qml was added twice - Autotune.cpp was not added to CMakeLists.txt

hamishwillee · 2021-11-10T05:15:07Z

@batinkov @bresch Did you guys already have a plan for the "autotuning guide" (referred to in the autotuning prompt)?

Here are some questions/things that spring to mind

What vehicles can be auto-tuned?
- Sounds like you can tune fixed wing or multicopter, or VTOL in either mode. Correct?
- Can you tune VTOL transition, boat, submarines, blimps? My "guess" is that you can tune the rate and attitude controllers on anything that has them (correct)?, but I "guess" that maybe you don't have these on a boat say?
Does this just work for PX4 or does the same UI work for ArduPilot?
What happens if the autotuning isn't supported?
- It sounds like autotuning will be attempted and it fails for any reason you'll get a notification.
- What form does that notification take?
Can this be tested in Gazebo?
- I tried and the UI isn't showing up in QGC daily today but wanted to be sure.
Tuning has to be done in the air. Is there any particular height recommendation? i.e. how much variation in height might you expect during tuning?
It looks like you have to land and disarm to apply the autotuning parameters - is that correct?
How long does it normally take?
- Looks like about 30 seconds in the video
- Does it take longer for different airframes?
How violent is it - i.e. are there frames that are likely to take damage from it?
Would I be correct that the MAVLink protocol is to simply send MAV_CMD_DO_AUTOTUNE_ENABLE to start enabling? Over this you have layered progress notification using STATUSTEXT?
- YOu could perhaps also have done this with https://mavlink.io/en/services/command.html#long_running_commands
What does it mean to say that autotuning might fail for a particular frame - i.e. is this because the

@bresch There are three tuning guides here: http://docs.px4.io/master/en/config_fw/ and a couple here http://docs.px4.io/master/en/config_mc/

Thoughts on how best to update these? It might be worth starting the relevant ones with a section explaining autotuning and pointing to the bits you still have to do manually.

My concern here is that at the moment we support "autotuning" on rate and attitude controllers. However it isn't clear if and when you need to tune position and velocity controllers, and what will happen if you don't. Is autotuning enough that most users can go on holiday, job done?

bresch · 2021-11-10T14:35:03Z

Before starting, let me mention that I'm planning to port the following guide to the PX4 doc:
https://docs.auterion.com/skynode/advanced-configuration/controllers-auto-tuning

What vehicles can be auto-tuned?
Sounds like you can tune fixed wing or multicopter, or VTOL in either mode. Correct?

Correct

Can you tune VTOL transition, boat, submarines, blimps? My "guess" is that you can tune the rate and attitude controllers on anything that has them (correct)?, but I "guess" that maybe you don't have these on a boat say?

VTOL transition: no, just tune MC and FW modes independently
boat, submarines, blimps: as you said, if they have a rate controller, we should be able to enable it, but currently, it's only interfaced for FW and MC, so no, it won't works atm.

Does this just work for PX4 or does the same UI work for ArduPilot?

I never tried

What happens if the autotuning isn't supported?
It sounds like autotuning will be attempted and it fails for any reason you'll get a notification.
What form does that notification take?

Yes, it should return "Autotuning: ack error"

Can this be tested in Gazebo?

Yes

I tried and the UI isn't showing up in QGC daily today but wanted to be sure.

That's strange, I would need to check

Tuning has to be done in the air. Is there any particular height recommendation? i.e. how much variation in height might you expect during tuning?

The drone can fly in alt/pos/auto modes during the auto-tuning, so the altitude shouldn't vary much. Also, if in pos/auto mode, the horizontal displacement should be small. In alt mode, it'll drift with wind and manual correction attempts will abort auto-tuning at the moment, this is still something we can improve if needed.

It looks like you have to land and disarm to apply the autotuning parameters - is that correct?

You can define that behavior in the [MC|FW]_AT_APPLY parameters. You can then choose to apply them in air or after disarm. For MC, apply after disarm is generally safer since you can then takeoff carefully (default behavior for MC). For FW it's better to apply in air because you'll get some time to recover if the tuning isn't great (default behavior for FW); furthermore, there is a logic to restore the previous parameters if the energy of the actuator signal is too large with the new gains.

How long does it normally take?
Looks like about 30 seconds in the video

The time is 5s-20s (aborted if did not converge in 20s) per axis + 2s pause between each axis + 4s of testing if the new gains are applied in air. So, minimum of 21s (= 3x5+3x2) without in-air test, 25s with in-air test and a worst case of 70s (if each axis takes exactly 20s to converge). Given the flight we've made so far, it usually takes no more than 40s.

Does it take longer for different airframes?
In FW mode, the axis to be tuned can be selected, so a user could make a run to just tune the roll axis for example. In that case, it will take less time. Other than that, it cannot take more than the worst case 70s.

How violent is it - i.e. are there frames that are likely to take damage from it?

It isn't violent on a standard MC but some larger VTOL planes in MC mode might not like large kicks, this is why the amplitude of the injected control signal (required for system identification) can be adjusted via a parameter (MC_AT_SYSID_AMP and FW_AT_SYSID_AMP) if needed.

Would I be correct that the MAVLink protocol is to simply send MAV_CMD_DO_AUTOTUNE_ENABLE to start enabling?

Yes

Over this you have layered progress notification using STATUSTEXT?

AFAIK, the command is sent periodically and the status is returned in the progress field of COMMAND_ACK

You could perhaps also have done this with https://mavlink.io/en/services/command.html#long_running_commands
What does it mean to say that autotuning might fail for a particular frame

The mathematical model used to estimate the dynamics of the drone assumes this it is a linear system with no coupling between the axes (SISO) and with a limited complexity (2 poles and 2 zeros). If the real drone is too far from those conditions, the model will not be able to represent the real dynamics of the drone.

hamishwillee · 2021-11-10T22:39:23Z

Thanks @bresch

Just my two bits, but I find it a little odd that the autotune button is hidden inside the rate/attitude controller sections. I understand why, but were it me I would

Enable it by default on frames where we think it is supported
Remove it altogether on flight stacks and frames where we know it is not supported
Maybe turn off the airmode option while autotune is enabled
Have it at the top outside above the tabs as the first thing people see - possibly with slight greying of the tabs that it affects when it is turned on - indicating that the other tabs still need to be looked at.

Why? Well this is what most people will want to do and right now it isn't obvious that the autotune buttons on the tabs are linked. Maybe get a UX designer to comment.

AFAIK, the command is sent periodically and the status is returned in the progress field of COMMAND_ACK

That's one way of implementing a client side the long running protocol (LRP). Essentially the LRP says you send a command, it acks periodically with progress until it completes, then it acks with acknowledged. If you ping it with the same message while in progress you get the progress ACK resent - which is what happens here. But you're ignoring the final ACK and just using the last received progress being 100% or disarmed to indicate completion.

Can you try out the gazebo? I tried fresh PX4 master and QGC daily today. The autotune button did not appear for multicopter. For plane the PID section did not appear.

hamishwillee · 2021-11-10T23:36:07Z

This won't work on ArduPilot, and that is a problem. They interpret this command as "enable autotune". That forces the fixed wing vehicle into an autotune mode. It ACKs with accepted immediately and completes - there is no progress update.

While I prefer the PX4 approach, doing something different than the intent is non-standard. It means that a GCS has to do something different for every single flight stack. No blame attributes here - the current command does not document it's assumptions.

Further, this implementation is not technically correct as per the long running protocol (though I have added mavlink/mavlink-devguide#405 to make it so if possible). Essentially the flight stack is supposed to emit the current progress - not the ground station poll. As pointed out, polling is potentially risky, particularly if you poll when the autotune has finished, because you will kick it off again.

Upshot, if we're going to do this, we probably need to agree a new command and document the precise semantics. Before the next release. Thoughts - @julianoes @DonLakeFlyer ?

bresch · 2021-11-11T09:19:23Z

Can you try out the gazebo? I tried fresh PX4 master and QGC daily today. The autotune button did not appear for multicopter.

It tried QGC daily in jMavSim and I also don't get the autotuning UI. @batinkov Could you please check what's happening?

batinkov · 2021-11-18T07:14:08Z

@bresch I don't know why you don't get the Autotune interface. I get it and it works as expected with PX4 (proprietary) and PX4 (open source) and gazebo_typhoon_h480, px4_sitl gazebo_standard_vtol, jmavsim, and px4_sitl gazebo. All possible combinations give me the interface and Autotune always works. Can someone else confirm they don't get the Autotune interface?

batinkov · 2021-11-18T07:21:58Z

@DonLakeFlyer I saw your comments up and I haven't forgotten about them. It's pretty crazy lately at work and that's why I'm delaying those fixes.

batinkov · 2021-11-18T07:23:30Z

Finally I'm not the right person to judge about MAVLink but it looks @hamishwillee is right about the Autotune implementation in PX4. Perhaps we need to change the polling mechanism in PX4 implementation and then QGC needs to change accordingly.

hamishwillee · 2021-11-18T08:08:03Z

Me, I don't get the autotune interface. Are you using ubuntu and current gazebo?

batinkov · 2021-11-18T08:12:20Z

@hamishwillee I'm using Ubuntu 18.04 LTS and gazebo 9.19.0.

hamishwillee · 2021-11-18T21:50:47Z

@batinkov How? QGC doesn't work for me on Ubuntu 18.04.
From https://docs.qgroundcontrol.com/master/en/getting_started/download_and_install.html#ubuntu

hamishwillee · 2021-11-19T03:55:31Z

@batinkov @bresch I did a completely clean install of PX4 and QGC daily on Ubuntu 20.04 and I am now seeing the autotune button!

DonLakeFlyer · 2021-11-19T17:58:21Z

Ramon fixed the 20.04 requirement a few days back. With that the docs are out of data and 18.04 is fine now.

mrpollo · 2021-11-24T16:59:20Z

@hamishwillee @DonLakeFlyer mavlink/qgc-user-guide#306

batinkov force-pushed the AMC-2336 branch 2 times, most recently from 35b3c08 to 445e33d Compare October 13, 2021 15:00

batinkov force-pushed the AMC-2336 branch from 445e33d to 1a59824 Compare October 18, 2021 13:36

batinkov force-pushed the AMC-2336 branch 2 times, most recently from 3a54b64 to 6cc71ff Compare October 19, 2021 11:17

batinkov marked this pull request as ready for review October 19, 2021 11:24

bresch reviewed Oct 19, 2021

View reviewed changes

batinkov force-pushed the AMC-2336 branch 2 times, most recently from a30b412 to d2439aa Compare October 19, 2021 15:24

Autotune enabled in QGC.

81410c9

batinkov force-pushed the AMC-2336 branch from d2439aa to 81410c9 Compare October 21, 2021 17:24

Autotune enabled in QGC - some tweaks to address comments after code …

e58bee6

…review.

batinkov requested review from bkueng and DonLakeFlyer October 22, 2021 10:06

bresch approved these changes Oct 22, 2021

View reviewed changes

bkueng approved these changes Oct 25, 2021

View reviewed changes

AMC-2336 - Small fix after code review: spaces => tab

e167dbc

bkueng merged commit 17cb9f4 into mavlink:master Oct 25, 2021

batinkov deleted the AMC-2336 branch October 25, 2021 11:26

DonLakeFlyer reviewed Oct 25, 2021

View reviewed changes

kistlin added a commit to kistlin/qgroundcontrol that referenced this pull request Nov 1, 2021

add missing source files to CMakeLists.txt

abf64c6

files added in mavlink#9904 - AutotuneUI.qml was added twice - Autotune.cpp was not added to CMakeLists.txt

kistlin mentioned this pull request Nov 1, 2021

add missing source files to CMakeLists.txt #9973

Merged

patrickelectric pushed a commit that referenced this pull request Nov 1, 2021

add missing source files to CMakeLists.txt

a8bdc41

files added in #9904 - AutotuneUI.qml was added twice - Autotune.cpp was not added to CMakeLists.txt

hamishwillee mentioned this pull request Nov 10, 2021

Long running commands - receiving same command is request for progress update mavlink/mavlink-devguide#405

Closed

hamishwillee mentioned this pull request Nov 24, 2021

Autotune docs PX4/PX4-user_guide#1661

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Autotune enabled in GQC. #9904

Autotune enabled in GQC. #9904

batinkov commented Oct 7, 2021 •

edited

Loading

batinkov commented Oct 7, 2021

bkueng commented Oct 19, 2021

bresch Oct 19, 2021 •

edited

Loading

batinkov Oct 22, 2021

bresch commented Oct 19, 2021

batinkov commented Oct 22, 2021

bresch commented Oct 22, 2021

bkueng Oct 25, 2021

batinkov Oct 25, 2021 •

edited

Loading

DonLakeFlyer left a comment

DonLakeFlyer Oct 25, 2021

DonLakeFlyer Oct 25, 2021 •

edited

Loading

hamishwillee Nov 10, 2021 •

edited

Loading

bkueng Nov 10, 2021

DonLakeFlyer Oct 25, 2021 •

edited

Loading

hamishwillee commented Nov 10, 2021 •

edited

Loading

bresch commented Nov 10, 2021

hamishwillee commented Nov 10, 2021 •

edited

Loading

hamishwillee commented Nov 10, 2021 •

edited

Loading

bresch commented Nov 11, 2021 •

edited

Loading

batinkov commented Nov 18, 2021

batinkov commented Nov 18, 2021 •

edited

Loading

batinkov commented Nov 18, 2021

hamishwillee commented Nov 18, 2021

batinkov commented Nov 18, 2021

hamishwillee commented Nov 18, 2021

hamishwillee commented Nov 19, 2021

DonLakeFlyer commented Nov 19, 2021

mrpollo commented Nov 24, 2021

	_autotuneStatus = tr("Autotune: Vehicle failed");
	_autotuneStatus = tr("Autotune: Failed");

		@@ -386,6 +387,8 @@ void Vehicle::_commonInit()

		_objectAvoidance = new VehicleObjectAvoidance(this, this);

		_autotune = _firmwarePlugin->createAutotune(this);

Autotune enabled in GQC. #9904

Autotune enabled in GQC. #9904

Conversation

batinkov commented Oct 7, 2021 • edited Loading

batinkov commented Oct 7, 2021

bkueng commented Oct 19, 2021

bresch Oct 19, 2021 • edited Loading

Choose a reason for hiding this comment

batinkov Oct 22, 2021

Choose a reason for hiding this comment

bresch commented Oct 19, 2021

batinkov commented Oct 22, 2021

bresch commented Oct 22, 2021

bkueng Oct 25, 2021

Choose a reason for hiding this comment

batinkov Oct 25, 2021 • edited Loading

Choose a reason for hiding this comment

DonLakeFlyer left a comment

Choose a reason for hiding this comment

DonLakeFlyer Oct 25, 2021

Choose a reason for hiding this comment

DonLakeFlyer Oct 25, 2021 • edited Loading

Choose a reason for hiding this comment

hamishwillee Nov 10, 2021 • edited Loading

Choose a reason for hiding this comment

bkueng Nov 10, 2021

Choose a reason for hiding this comment

DonLakeFlyer Oct 25, 2021 • edited Loading

Choose a reason for hiding this comment

hamishwillee commented Nov 10, 2021 • edited Loading

bresch commented Nov 10, 2021

hamishwillee commented Nov 10, 2021 • edited Loading

hamishwillee commented Nov 10, 2021 • edited Loading

bresch commented Nov 11, 2021 • edited Loading

batinkov commented Nov 18, 2021

batinkov commented Nov 18, 2021 • edited Loading

batinkov commented Nov 18, 2021

hamishwillee commented Nov 18, 2021

batinkov commented Nov 18, 2021

hamishwillee commented Nov 18, 2021

hamishwillee commented Nov 19, 2021

DonLakeFlyer commented Nov 19, 2021

mrpollo commented Nov 24, 2021

batinkov commented Oct 7, 2021 •

edited

Loading

bresch Oct 19, 2021 •

edited

Loading

batinkov Oct 25, 2021 •

edited

Loading

DonLakeFlyer Oct 25, 2021 •

edited

Loading

hamishwillee Nov 10, 2021 •

edited

Loading

DonLakeFlyer Oct 25, 2021 •

edited

Loading

hamishwillee commented Nov 10, 2021 •

edited

Loading

hamishwillee commented Nov 10, 2021 •

edited

Loading

hamishwillee commented Nov 10, 2021 •

edited

Loading

bresch commented Nov 11, 2021 •

edited

Loading

batinkov commented Nov 18, 2021 •

edited

Loading