feat: add API to save messages (#5606)

> _took quite some time until i found the time to finish this PR and to
find a time window that does not disturb other developments too much,
but here we are:_

this PR enables UI to improve "Saved messages" hugely, bringing it on
WhatsApp's "Starred Messages" or Telegram's "Saved Messages" level. with
this PR, UIs can add the following functionality with few effort ([~100
loc on iOS](deltachat/deltachat-ios#2527)):

- add a "Save" button in the messages context menu, allowing to save a
message
- show directly in the chat if a message was saved, eg. by a little star
★
- in "Saved Messages", show the message in its context - so with author,
avatar, date and keep incoming/outgoing state
- in "Saved Messages", a button to go to the original message can be
added
- if the original message was deleted, one can still go to the original
chat

these features were often requested, in the forum, but also in many
one-to-one discussions, recently at the global gathering.

moreover, in contrast to the old method with "forward to saved", no
traffic is wasted - the messages are saved locally, and only a small
sync messages is sent around

this is how it looks out on iOS:

<img width="260" alt="Screenshot 2025-01-17 at 00 02 51"
src="https://github.com/user-attachments/assets/902741b6-167f-4af1-9f9a-bd0439dd20d0"
/> &nbsp; &nbsp; <img width="353" alt="Screenshot 2025-01-17 at 00 05
33"
src="https://github.com/user-attachments/assets/97eceb79-6e43-4a89-9792-2e077e7141cd"
/>

technically, still a copy is done from the original message (with
already now deduplicated blobs), so that things work nicely with
deletion modes; eg. you can save an important message and preserve it
from deletion that way.

jsonrpc can be done in a subsequent PR, i was implementing the UI on iOS
where that was not needed (and most API were part of message object that
is not in use in jsonrpc atm)

@hpk42 the forward issue we discussed earller that day is already solved
(first implementation did not had an explict save_msgs() but was using
forward_msgs(SELF) as saving - with the disadvantage that forwarding to
SELF is not working, eg. if one wants the old behaviour) acutally, that
made the PR a lot nicer, as now very few existing code is changed

<details>
<summary>previous considerations and abandoned things</summary>

while working on this PR, there was also the idea to just set a flag
“starred” in the message table and not copy anything. however, while
tempting, that would result in more complexity, questions and drawbacks
in UI:

- delete-message, delete-chat, auto-deletion, clear-chat would raise
questions - what do do with the “Starred”? having a copy in “Saved
Messages” does not raise this question
- newly saved messages appear naturally as “new” in “Saved Messages”,
simply setting a flag would show them somewhere in between - unless we
do additional effort
- “Saved Messages” already has its place in the UI - and allows to
_directly_ save things there as well - not easily doable with “starring”
- one would need to re-do many things that already exist in “Saved
Messages”, at least in core
- one idea to solve some of the issues could be to have “Starred” as
well as “Saved Messages” - however, that would irritate user, one would
remember exactly what was done with a message, and understand the fine
differences

whatsapp does this “starred”, btw, so when original is deleted, starred
is deleted as well. Telegram does things similar to us, Signal does
nothing. Whatsapp has a per-chat view for starred messages - if needed,
we could do sth. like that as well, however, let’s first see how far the
“all view” goes (in contrast to whatsapp, we have profiles for
separation as well)

for the wording: “saving” is what we’re doing here, this is much more on
point as “starring” - which is more the idea of a “bookmark”, and i
think, whatsapp uses this wording to not raise false expectations
(still, i know of ppl that were quite upset that a “starred” message was
deleted when eg. the chat was cleared to save some memory)

wrt webxdc app updates: options that come into mind were: _empty_ (as
now), _snapshot_ (copy all updates) or _shortcut_ (always open
original). i am not sure what the best solution is, the easiest was
_empty_, so i went for that, as it is (a) obvious, and what is already
done with forwarding and (b) the original is easy to open now (in
contrast to forwarding).
so, might totally be that we need or want to tweak things here, but i
would leave that outside the first iteration, things are not worsened in
that area

wrt reactions: as things are detached, similar to webxdc updates, we do
not not to show the original reactions - that way, one can use reactions
for private markers (telegram is selling that feature :)

to the icon: a disk or a bookmark might be other options, but the star
is nicer and also know from other apps - and anyways a but vague UX
wise. so i think, it is fine

finally, known issue is that if a message was saved that does not exist
on another device, it does not get there. i think, that issue is a weak
one, and can be ignored mostly, most times, user will save messages soon
after receiving, and if on some devices auto-deletion is done, it is
maybe not even expected to have suddenly another copy there

</details>

EDIT: once this is merged, detailed issues about what to do should be
filed for android/desktop (however, they do not have urgency to adapt,
things will continue working as is)

---------

Co-authored-by: Hocuri <[email protected]>

deltachat-ffi/deltachat.h

@@ -1974,6 +1974,36 @@ void            dc_delete_msgs               (dc_context_t* context, const uint3
 void            dc_forward_msgs              (dc_context_t* context, const uint32_t* msg_ids, int msg_cnt, uint32_t chat_id);
 
 
+/**
+ * Save a copy of messages in "Saved Messages".
+ *
+ * In contrast to forwarding messages,
+ * information as author, date and origin are preserved.
+ * The action completes locally, so "Saved Messages" do not show sending errors in case one is offline.
+ * Still, a sync message is emitted, so that other devices will save the same message,
+ * as long as not deleted before.
+ *
+ * To check if a message was saved, use dc_msg_get_saved_msg_id(),
+ * UI may show an indicator and offer an "Unsave" instead of a "Save" button then.
+ *
+ * The other way round, from inside the "Saved Messages" chat,
+ * UI may show the indicator and "Unsave" button checking dc_msg_get_original_msg_id()
+ * and offer a button to go the original message.
+ *
+ * "Unsave" is done by deleting the saved message.
+ * Webxdc updates are not copied on purpose.
+ *
+ * For performance reasons, esp. when saving lots of messages,
+ * UI should call this function from a background thread.
+ *
+ * @memberof dc_context_t
+ * @param context The context object.
+ * @param msg_ids An array of uint32_t containing all message IDs that should be saved.
+ * @param msg_cnt The number of messages IDs in the msg_ids array.
+ */
+void            dc_save_msgs                 (dc_context_t* context, const uint32_t* msg_ids, int msg_cnt);
+
+
 /**
  * Resend messages and make information available for newly added chat members.
  * Resending sends out the original message, however, recipients and webxdc-status may differ.
@@ -4868,6 +4898,33 @@ dc_msg_t*       dc_msg_get_quoted_msg         (const dc_msg_t* msg);
 dc_msg_t*       dc_msg_get_parent             (const dc_msg_t* msg);
 
 
+/**
+ * Get original message ID for a saved message from the "Saved Messages" chat.
+ *
+ * Can be used by UI to show a button to go the original message
+ * and an option to "Unsave" the message.
+ *
+ * @param msg The message object. Usually, this refers to a a message inside "Saved Messages".
+ * @return The message ID of the original message.
+ *     0 if the given message object is not a "Saved Message"
+ *     or if the original message does no longer exist.
+ */
+uint32_t        dc_msg_get_original_msg_id    (const dc_msg_t* msg);
+
+
+/**
+ * Check if a message was saved and return its ID inside "Saved Messages".
+ *
+ * Deleting the returned message will un-save the message.
+ * The state "is saved" can be used to show some icon to indicate that a message was saved.
+ *
+ * @param msg The message object. Usually, this refers to a a message outside "Saved Messages".
+ * @return The message ID inside "Saved Messages", if any.
+ *     0 if the given message object is not saved.
+ */
+uint32_t        dc_msg_get_saved_msg_id     (const dc_msg_t* msg);
+
+
 /**
  * Force the message to be sent in plain text.
  *

deltachat-ffi/src/lib.rs

@@ -1979,6 +1979,26 @@ pub unsafe extern "C" fn dc_forward_msgs(
     })
 }
 
+#[no_mangle]
+pub unsafe extern "C" fn dc_save_msgs(
+    context: *mut dc_context_t,
+    msg_ids: *const u32,
+    msg_cnt: libc::c_int,
+) {
+    if context.is_null() || msg_ids.is_null() || msg_cnt <= 0 {
+        eprintln!("ignoring careless call to dc_save_msgs()");
+        return;
+    }
+    let msg_ids = convert_and_prune_message_ids(msg_ids, msg_cnt);
+    let ctx = &*context;
+
+    block_on(async move {
+        chat::save_msgs(ctx, &msg_ids[..])
+            .await
+            .unwrap_or_log_default(ctx, "Failed to save message")
+    })
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn dc_resend_msgs(
     context: *mut dc_context_t,
@@ -3980,6 +4000,48 @@ pub unsafe extern "C" fn dc_msg_get_parent(msg: *const dc_msg_t) -> *mut dc_msg_
     }
 }
 
+#[no_mangle]
+pub unsafe extern "C" fn dc_msg_get_original_msg_id(msg: *const dc_msg_t) -> u32 {
+    if msg.is_null() {
+        eprintln!("ignoring careless call to dc_msg_get_original_msg_id()");
+        return 0;
+    }
+    let ffi_msg: &MessageWrapper = &*msg;
+    let context = &*ffi_msg.context;
+    block_on(async move {
+        ffi_msg
+            .message
+            .get_original_msg_id(context)
+            .await
+            .context("failed to get original message")
+            .log_err(context)
+            .unwrap_or_default()
+            .map(|id| id.to_u32())
+            .unwrap_or(0)
+    })
+}
+
+#[no_mangle]
+pub unsafe extern "C" fn dc_msg_get_saved_msg_id(msg: *const dc_msg_t) -> u32 {
+    if msg.is_null() {
+        eprintln!("ignoring careless call to dc_msg_get_saved_msg_id()");
+        return 0;
+    }
+    let ffi_msg: &MessageWrapper = &*msg;
+    let context = &*ffi_msg.context;
+    block_on(async move {
+        ffi_msg
+            .message
+            .get_saved_msg_id(context)
+            .await
+            .context("failed to get original message")
+            .log_err(context)
+            .unwrap_or_default()
+            .map(|id| id.to_u32())
+            .unwrap_or(0)
+    })
+}
+
 #[no_mangle]
 pub unsafe extern "C" fn dc_msg_force_plaintext(msg: *mut dc_msg_t) {
     if msg.is_null() {

src/chat.rs

@@ -4209,6 +4209,80 @@ pub async fn forward_msgs(context: &Context, msg_ids: &[MsgId], chat_id: ChatId)
     Ok(())
 }
 
+/// Save a copy of the message in "Saved Messages"
+/// and send a sync messages so that other devices save the message as well, unless deleted there.
+pub async fn save_msgs(context: &Context, msg_ids: &[MsgId]) -> Result<()> {
+    for src_msg_id in msg_ids {
+        let dest_rfc724_mid = create_outgoing_rfc724_mid();
+        let src_rfc724_mid = save_copy_in_self_talk(context, src_msg_id, &dest_rfc724_mid).await?;
+        context
+            .add_sync_item(SyncData::SaveMessage {
+                src: src_rfc724_mid,
+                dest: dest_rfc724_mid,
+            })
+            .await?;
+    }
+    context.send_sync_msg().await?;
+    Ok(())
+}
+
+/// Saves a copy of the given message in "Saved Messages" using the given RFC724 id.
+/// To allow UIs to have a "show in context" button,
+/// the copy contains a reference to the original message
+/// as well as to the original chat in case the original message gets deleted.
+/// Returns data needed to add a `SaveMessage` sync item.
+pub(crate) async fn save_copy_in_self_talk(
+    context: &Context,
+    src_msg_id: &MsgId,
+    dest_rfc724_mid: &String,
+) -> Result<String> {
+    let dest_chat_id = ChatId::create_for_contact(context, ContactId::SELF).await?;
+    let mut msg = Message::load_from_db(context, *src_msg_id).await?;
+    msg.param.remove(Param::Cmd);
+    msg.param.remove(Param::WebxdcDocument);
+    msg.param.remove(Param::WebxdcDocumentTimestamp);
+    msg.param.remove(Param::WebxdcSummary);
+    msg.param.remove(Param::WebxdcSummaryTimestamp);
+
+    if !msg.original_msg_id.is_unset() {
+        bail!("message already saved.");
+    }
+
+    let copy_fields = "from_id, to_id, timestamp_sent, timestamp_rcvd, type, txt, txt_raw, \
+                             mime_modified, mime_headers, mime_compressed, mime_in_reply_to, subject, msgrmsg";
+    let row_id = context
+        .sql
+        .insert(
+            &format!(
+                "INSERT INTO msgs ({copy_fields}, chat_id, rfc724_mid, state, timestamp, param, starred) \
+                            SELECT {copy_fields}, ?, ?, ?, ?, ?, ? \
+                            FROM msgs WHERE id=?;"
+            ),
+            (
+                dest_chat_id,
+                dest_rfc724_mid,
+                if msg.from_id == ContactId::SELF {
+                    MessageState::OutDelivered
+                } else {
+                    MessageState::InSeen
+                },
+                create_smeared_timestamp(context),
+                msg.param.to_string(),
+                src_msg_id,
+                src_msg_id,
+            ),
+        )
+        .await?;
+    let dest_msg_id = MsgId::new(row_id.try_into()?);
+
+    context.emit_msgs_changed(msg.chat_id, *src_msg_id);
+    context.emit_msgs_changed(dest_chat_id, dest_msg_id);
+    chatlist_events::emit_chatlist_changed(context);
+    chatlist_events::emit_chatlist_item_changed(context, dest_chat_id);
+
+    Ok(msg.rfc724_mid)
+}
+
 /// Resends given messages with the same Message-ID.
 ///
 /// This is primarily intended to make existing webxdcs available to new chat members.
@@ -4703,7 +4777,7 @@ mod tests {
     use crate::chatlist::get_archived_cnt;
     use crate::constants::{DC_GCL_ARCHIVED_ONLY, DC_GCL_NO_SPECIALS};
     use crate::headerdef::HeaderDef;
-    use crate::message::delete_msgs;
+    use crate::message::{delete_msgs, MessengerMessage};
     use crate::receive_imf::receive_imf;
     use crate::test_utils::{sync, TestContext, TestContextManager, TimeShiftFalsePositiveNote};
     use strum::IntoEnumIterator;
@@ -6836,6 +6910,136 @@ mod tests {
         Ok(())
     }
 
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_save_msgs() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let bob = TestContext::new_bob().await;
+        let alice_chat = alice.create_chat(&bob).await;
+
+        let sent = alice.send_text(alice_chat.get_id(), "hi, bob").await;
+        let sent_msg = Message::load_from_db(&alice, sent.sender_msg_id).await?;
+        assert!(sent_msg.get_saved_msg_id(&alice).await?.is_none());
+        assert!(sent_msg.get_original_msg_id(&alice).await?.is_none());
+
+        let self_chat = alice.get_self_chat().await;
+        save_msgs(&alice, &[sent.sender_msg_id]).await?;
+
+        let saved_msg = alice.get_last_msg_in(self_chat.id).await;
+        assert_ne!(saved_msg.get_id(), sent.sender_msg_id);
+        assert!(saved_msg.get_saved_msg_id(&alice).await?.is_none());
+        assert_eq!(
+            saved_msg.get_original_msg_id(&alice).await?.unwrap(),
+            sent.sender_msg_id
+        );
+        assert_eq!(saved_msg.get_text(), "hi, bob");
+        assert!(!saved_msg.is_forwarded()); // UI should not flag "saved messages" as "forwarded"
+        assert_eq!(saved_msg.is_dc_message, MessengerMessage::Yes);
+        assert_eq!(saved_msg.get_from_id(), ContactId::SELF);
+        assert_eq!(saved_msg.get_state(), MessageState::OutDelivered);
+        assert_ne!(saved_msg.rfc724_mid(), sent_msg.rfc724_mid());
+
+        let sent_msg = Message::load_from_db(&alice, sent.sender_msg_id).await?;
+        assert_eq!(
+            sent_msg.get_saved_msg_id(&alice).await?.unwrap(),
+            saved_msg.id
+        );
+        assert!(sent_msg.get_original_msg_id(&alice).await?.is_none());
+
+        let rcvd_msg = bob.recv_msg(&sent).await;
+        let self_chat = bob.get_self_chat().await;
+        save_msgs(&bob, &[rcvd_msg.id]).await?;
+        let saved_msg = bob.get_last_msg_in(self_chat.id).await;
+        assert_ne!(saved_msg.get_id(), rcvd_msg.id);
+        assert_eq!(
+            saved_msg.get_original_msg_id(&bob).await?.unwrap(),
+            rcvd_msg.id
+        );
+        assert_eq!(saved_msg.get_text(), "hi, bob");
+        assert!(!saved_msg.is_forwarded());
+        assert_eq!(saved_msg.is_dc_message, MessengerMessage::Yes);
+        assert_ne!(saved_msg.get_from_id(), ContactId::SELF);
+        assert_eq!(saved_msg.get_state(), MessageState::InSeen);
+        assert_ne!(saved_msg.rfc724_mid(), rcvd_msg.rfc724_mid());
+
+        // delete original message
+        delete_msgs(&bob, &[rcvd_msg.id]).await?;
+        let saved_msg = Message::load_from_db(&bob, saved_msg.id).await?;
+        assert!(saved_msg.get_original_msg_id(&bob).await?.is_none());
+
+        // delete original chat
+        rcvd_msg.chat_id.delete(&bob).await?;
+        let msg = Message::load_from_db(&bob, saved_msg.id).await?;
+        assert!(msg.get_original_msg_id(&bob).await?.is_none());
+
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_saved_msgs_not_added_to_shared_chats() -> Result<()> {
+        let mut tcm = TestContextManager::new();
+        let alice = tcm.alice().await;
+        let bob = tcm.bob().await;
+
+        let msg = tcm.send_recv_accept(&alice, &bob, "hi, bob").await;
+
+        let self_chat = bob.get_self_chat().await;
+        save_msgs(&bob, &[msg.id]).await?;
+        let msg = bob.get_last_msg_in(self_chat.id).await;
+        let contact = Contact::get_by_id(&bob, msg.get_from_id()).await?;
+        assert_eq!(contact.get_addr(), "[email protected]");
+
+        let shared_chats = Chatlist::try_load(&bob, 0, None, Some(contact.id)).await?;
+        assert_eq!(shared_chats.len(), 1);
+        assert_eq!(
+            shared_chats.get_chat_id(0).unwrap(),
+            bob.get_chat(&alice).await.id
+        );
+
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_forward_from_saved_to_saved() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let bob = TestContext::new_bob().await;
+        let sent = alice.send_text(alice.create_chat(&bob).await.id, "k").await;
+
+        bob.recv_msg(&sent).await;
+        let orig = bob.get_last_msg().await;
+        let self_chat = bob.get_self_chat().await;
+        save_msgs(&bob, &[orig.id]).await?;
+        let saved1 = bob.get_last_msg().await;
+        assert_eq!(
+            saved1.get_original_msg_id(&bob).await?.unwrap(),
+            sent.sender_msg_id
+        );
+        assert_ne!(saved1.from_id, ContactId::SELF);
+
+        forward_msgs(&bob, &[saved1.id], self_chat.id).await?;
+        let saved2 = bob.get_last_msg().await;
+        assert!(saved2.get_original_msg_id(&bob).await?.is_none(),);
+        assert_eq!(saved2.from_id, ContactId::SELF);
+
+        Ok(())
+    }
+
+    #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+    async fn test_save_from_saved_to_saved_failing() -> Result<()> {
+        let alice = TestContext::new_alice().await;
+        let bob = TestContext::new_bob().await;
+        let sent = alice.send_text(alice.create_chat(&bob).await.id, "k").await;
+
+        bob.recv_msg(&sent).await;
+        let orig = bob.get_last_msg().await;
+        save_msgs(&bob, &[orig.id]).await?;
+        let saved1 = bob.get_last_msg().await;
+
+        let result = save_msgs(&bob, &[saved1.id]).await;
+        assert!(result.is_err());
+
+        Ok(())
+    }
+
     #[tokio::test(flavor = "multi_thread", worker_threads = 2)]
     async fn test_resend_own_message() -> Result<()> {
         // Alice creates group with Bob and sends an initial message