Ambi Climate has a function where it can pick up other IR signals in the room so that you can continue to use your physical remote in conjunction with the app.
If the signal was successfully captured (requires correct pairing), your app will display the action taken with the physical remote and so you could be aware that your air conditioner or heat pump is no longer still under Ambi Climate's smart modes, hence not auto-adjusting.
Ambi Climate thinks you wish to control in manual mode so it display the setting you've done on your physical remote but will not attempt to resend a smart mode adjustment to switch it back to smart modes unless you manually switch it back to a smart mode or have set a timer that happens after the physical action.