Automating Translation File Uploads to Lokalise with GitHub Actions
Recently, I faced the intricate challenge of automating the upload of translation files to Lokalise using GitHub Actions. This journey involved multiple trials with different base64 encoding methods and countless refinements to achieve a seamless solution. Here’s a detailed account of the problem, my efforts, and the ultimate solution.
The Challenge
Automating the upload of a translations file to Lokalise via GitHub Actions required encoding the file in base64 and sending it through a POST request. The complexity lay in correctly formatting the data while managing GitHub secrets securely.
Initial Attempts
I tried directly encoding the file content and embedding it in the JSON payload within the GitHub Actions script. Here’s a snippet of the command I used:
FILE_CONTENT=$(base64 file.json | tr -d '\n')
curl --request POST \
--url "https://api.lokalise.com/api2/projects/$/files/upload" \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '
{
"data": '"$FILE_CONTENT"'
}
Error
Argument list too long." This error occurs when the encoded content is too large for the shell to handle directly
Inline Encoding in curl:
curl --request POST \
--url "https://api.lokalise.com/api2/projects/$/files/upload" \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '
{
"data": '"$(base64 file.json | tr -d '\n')"'
}
Error
Similar issue with argument length limitations and improper JSON formatting
I tried directly encoding the file content and embedding it in the JSON payload within the GitHub Actions script. Here’s a snippet of the command I used:
BASE64_CONTENT=$(base64 -i file.json -o encoded_file.txt)
curl --request POST \
--url "https://api.lokalise.com/api2/projects/$/files/upload" \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '
{
"data": '"$BASE64_CONTENT"'
}
However, this approach caused issues due to newline characters and incorrect escaping, leading to invalid JSON payloads
The Solution
The breakthrough came by writing the base64 output directly to a file and reading it back into the script.
I also switched from base64 to openssl base64, which gave me more flexibility when working with GitHub Actions.
Here’s the final approach that solved the issue:
- Encode the file and set the variable:
openssl base64 -A -in file.json -out encoded_file.json
BASE64_CONTENT=$(cat encoded_file.json)
- Create JSON Payload:
json_content=$(cat EOF
{
"data": "$BASE64_CONTENT",
"lang_iso": "en_US",
"filename": "index.json",
"use_automations": true,
"cleanup_mode": true,
"slashn_to_linebreak": true,
"replace_modified": true
}
EOF
)
echo "$json_content" > data.json
- Upload to Lokalise:
upload_response=$(curl --request POST \
--url "https://api.lokalise.com/api2/projects/$/files/upload" \
--header "X-Api-Token: $" \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '@data.json')
Finally, success.
With this now working and correctly uploading the payload, I was able to add some checks to ensure I download the correct translation job.
- Checking Upload Status:
process_id=$(echo $upload_response | jq -r '.process.process_id')
check_status() {
status_response=$(curl --request GET \
--url "https://api.lokalise.com/api2/projects/$/processes/$process_id" \
--header "X-Api-Token: $" \
--header "accept: application/json")
process_status=$(echo $status_response | jq -r '.process.status')
if [ "$process_status" = "finished" ]; then
return 0
else
return 1
fi
}
while ! check_status; do
sleep 2
done
Conclusion
Through persistence and numerous trials, I managed to automate the translation file uploads to Lokalise.
The process of base64 encoding in a CI/CD pipeline revealed the importance of handling large data and JSON formatting correctly. By outputting the encoded content to a file and reading it back, I bypassed the argument length limitations and achieved a successful upload.
For reference, this is the full GitHub Action I used:
name: Update Translations
on:
pull_request:
branches:
- main
jobs:
update-translations:
runs-on: ubuntu-latest
permissions:
contents: 'write'
id-token: 'write'
if: $
steps:
- name: Check Branch
run: echo $
- name: Checkout repository
uses: actions/checkout@v4
with:
ref: $
- name: Install jq
run: sudo apt-get install -y jq
- name: Upload file to Lokalise
run: |
openssl base64 -A -in file.json -out encoded_file.json
echo "Set Variable"
BASE64_CONTENT=$(cat encoded_file.json)
echo $BASE64_CONTENT
echo "Set JSON Content"
json_content=$(cat EOF
{
"data": "$BASE64_CONTENT",
"lang_iso": "en_US",
"filename": "index.json",
"use_automations": true,
"cleanup_mode": true,
"slashn_to_linebreak": true,
"replace_modified": true
}
EOF
)
echo "$json_content" > data.json
cat data.json
upload_response=$(curl --request POST \
--url "https://api.lokalise.com/api2/projects/$/files/upload" \
--header "X-Api-Token: $" \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '@data.json')
process_id=$(echo $upload_response | jq -r '.process.process_id')
echo "Process ID: $process_id"
# Function to check process status
check_status() {
status_response=$(curl --request GET \
--url "https://api.lokalise.com/api2/projects/$/processes/$process_id" \
--header "X-Api-Token: $" \
--header "accept: application/json")
process_status=$(echo $status_response | jq -r '.process.status')
echo "Current status: $process_status"
if [ "$process_status" = "finished" ]; then
return 0
else
return 1
fi
}
# Loop to check status every 10 seconds until it's finished
while ! check_status; do
echo "Waiting for process to finish..."
sleep 2
done
echo "Process finished."
- name: Run translation update
run: |
DOWNLOAD_URL="https://api.lokalise.com/api2/projects/$/files/download"
# Create a temporary directory to store the download link response
TEMP_DIR=$(mktemp -d)
DOWNLOAD_LINK_FILE="${TEMP_DIR}/download_link.json"
# Request the download link from Lokalise
curl --request POST \
--url $DOWNLOAD_URL \
--header "X-Api-Token: $" \
--header 'accept: application/json' \
--header 'content-type: application/json' \
--data '
{
"filter_langs": [
"en_GB",
"fr",
"de",
"it",
"tlh_KL",
"pl"
],
"format": "json",
"export_empty_as": "base",
"export_sort": "first_added",
"plural_format": "i18next_v4",
"indentation": "4sp",
"original_filenames": false,
"escape_percent": true
}
' > $DOWNLOAD_LINK_FILE
# Extract the download URL from the response
DOWNLOAD_URL=$(jq -r '.bundle_url' $DOWNLOAD_LINK_FILE)
# Download the translations file
curl --output "${TEMP_DIR}/translations.zip" $DOWNLOAD_URL
# Unzip the translations to the temporary directory
unzip -o "${TEMP_DIR}/translations.zip" -d "${TEMP_DIR}/translations"
# Copy the contents of the locale directory to the target translations directory
cp -r "${TEMP_DIR}/translations/locale/." ./translations
# Clean up the temporary directory
rm -rf $TEMP_DIR
rm data.json
rm encoded_file.json
# Commit changes
git config --global user.email "platform+benefexbot@benefex.co"
git config --global user.name "Benefex Bot"
git pull -q
git add ./translations
git commit -m "Update translations from Lokalise"
git push
