Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

android : Apply chat template #11059

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

Dhruvanand24
Copy link

@Dhruvanand24 Dhruvanand24 commented Jan 3, 2025

Description

This PR introduces foundational changes to enhance chat template functionality in the Llama Android project. While the core functionality has been implemented, integration with the main application flow is pending and will be addressed in future updates.

  1. Added JNI Support:

    • Integrated JNI for applying chat templates in llama-android.cpp.
    • This implementation processes a list of messages in a structured format for better templating capabilities.
  2. Updated LlamaAndroid.kt:

    • Added a new function to handle chat templates.
    • This function accepts a list of maps (List<Map<String, String>>) for structured message handling.
  3. Refactored MainViewModel.kt:

    • Modified messages to be a list instead of a string:
      var messages by mutableStateOf(listOf<Map<String, String>>())
  4. Pending Integration:

    • The new applyChatTemplate function is created but not yet invoked in the application flow.
    • Future updates will include logic to call this function as follows:
      llamaAndroid.send(llamaAndroid.applyChatTemplate(messages))

Benefits

  • Enhanced Structure: Enables structured message handling for better scalability and maintainability.
  • Increased Flexibility: Supports dynamic application of chat templates.
  • Future-Proof: Lays the groundwork for more sophisticated chat interactions.

Testing

  • Verified JNI integration for chat template processing.
  • Ensured compatibility with the updated MainViewModel.kt.
  • Basic functionality of applyChatTemplate tested with sample data.

To-Do

  • Integrate applyChatTemplate into the main application flow.
  • Add unit tests for applyChatTemplate in LlamaAndroid.kt.
  • Conduct integration testing for JNI enhancements.

Reviewer Notes

  • Feedback on structuring the pending integration logic is welcome.
  • Suggestions for improving the applyChatTemplate function are appreciated.
  • Insights on optimizing JNI interactions for performance would be valuable.

Created 2 more utility function- mapListToJSONString and format_chat.
Added Function to apply chat template in LLamaAndroid.kt
@github-actions github-actions bot added android Issues specific to Android examples labels Jan 3, 2025
@ngxson ngxson changed the title Apply chat template android : Apply chat template Jan 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
android Issues specific to Android examples
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant