How to work with ffmpeg tool and scripting with file names containing non ASCII characters.

R

rupeshforu3

Disciple
Hi I am Rupesh from India and I have a PC with windows 8 and open suse Linux installed. I have some MP4 video files of large size. I want to compress these files using ffmpeg command line tool not gui. I can't do this work because the file names consists of non ASCII characters.


There are 150 files to be converted. In the command prompt when I navigate to this directory and issue the command "ls -name" I am getting output as lines containing non ASCII characters with English characters. When I run the command "ls -name > names.txt" and open the text file in text editor I am able to see correct file names.


In the names.txt file one of the line consists of xxxxx_english_charecters_xxxxx.mp4. In this line xxxxx are non ASCII characters i mean characters from other non English language.


I have issued the following command

ffmpeg -i -y xxxxx_english_charecters_xxxxx.mp4 -c:v libx265 -b:v 400k -c:a aac -b:a 64k -ar 44100 output.mp4.



After issuing the above mentioned command I am getting error from ffmpeg command as below


File xxxxx_english_charecters_xxxxx.mp4 not found.


My complaint is that windows command prompt or linux terminal emulator are unable to display or interpret non ASCII characters.

I have written a small shell script for Linux as below


for i in *.mp4;
do name=`echo $i | cut -d'.' -f1`;
echo $name;
ffmpeg -i "$i" -c:v libx265 -b:v 400k -c:a aac -b:a 64k -ar 44100 "${name}_compressed.mp4";
done



After that I given execute permission to this script file and tried to execute

Upon running the above script in terminal emulator i am getting continuous errors from ffmpeg command as

File unknown
File unknown


After that I have written a batch script for windows as follows


@Echo off

setlocal EnableExtensions DisableDelayedExpansion

rem // Define constants here:
set "_SOURCE=E:\to_convert\new2"
set "_TARGET=F:\FFOutput"

rem // Change to source directory temporarily:
pushd "%_SOURCE%" || exit /B 1

rem // Enumerate source files, return paths relative to the source directory:

for /F "delims=" %%F in ('xcopy /L /S /I ".\*.mp4" "%_TARGET%" ^| find "."') do (
echo Currently converting "%%F"...
rem // Create destination directory, suppress error if it aready exists:
mkdir "%_TARGET%\%%F\.." 2> nul

rem // Perform actual file conversion, using paths relative to target directory:
ffmpeg -y -i "%%F" -c:v libx265 -b:v 400k -pass 2 -c:a aac -b:a 48k -ar 44100 "%_TARGET%\%%F\..\%%~nF.mp4"

)
echo Completed.
popd

endlocal

exit /B


After execution of this batch file also I am getting continuous errors from ffmpeg command as

File not found
File not found.

If you can't believe my words try to create an empty file with file name containing chinise characters in windows command prompt or Linux terminal emulator using command like touch. Definitely you will fail to create.


Even if you can't believe my words try to download or copy any file from internet with file name containing chinise characters. After that try to navigate to the directory downloaded in windows power shell and issue the ls command and definitely you are not going to see file you downloaded.


I have searched web for " unicode support in command prompt " " unicode support in terminal emulator " but I have not found any suitable solution.


Can anyone of you suggest how to work with command line tools like ffmpeg, touch and using these commands in shell scripting or windows command prompt and windows power shell.


Regards,
Rupesh.
 
I'm not sure whether ffmpeg supports unicode filenames. if it doesn't support, then you can try using DOS 8.3 filenames. But for using that, you need to enable(if it's not enabled already) that in registry and copy paste your files into the new folder. Then windows will generate DOS 8.3 file names for all the files, basically its a short file names without any unicode characters. After this you can use your scripts.
 
Hi I have copied script code from Linux tutorial and ffmpeg x265 help page. The ffmpeg x265 link is

https://trac.ffmpeg.org/wiki/Encode/H.265

I am listing the code as below

for i in *.mp4;
do name=`echo $i | cut -d'.' -f1`;
echo $name;
ffmpeg -y -i "$i" -c:v libx265 -x265-params pass=1 -an -f null /dev/null && \
ffmpeg -i input -c:v libx265 -x265-params pass=2 -c:a aac -b:a 48k -ar 44100 "${name}_compressed.mp4";
done

Fortunately ffmpeg command in Linux shell scripting is working properly and previously I have not noticed.

After execution of the above script i am able to see some processing progress of ffmpeg command in Linux terminal emulator but at finally it is showing error as " atleast one input must be specified ".

May I know what's wrong with the above script.

One small request is how to convert an x264 with mp4 file extension to x265 with mp4 file extension and at the same time the video resolution, frame rate of output file and input file must be same.

Suppose I have a source x264 file with file name as my_video.mp4 with video resolution as 1920*1080 and frame rate as 28 frames per second.

I want to convert it to x265 with file name as my_video.mp4 with video resolution as 1920*1080 and frame rate as 28 frames per second.

I have searched web and found that we must include -vtag hvc1 in the options of ffmpeg command.

Somewhere I found we must include -vtag hev1 in the options of ffmpeg command.

May I know what is the difference between hvc1 and hev1 and which is best.
 
Can you paste the names of a few files here with the unicode samples?

And BTW if your file names have space in between always enclose the file name variable in ".

for i in *.mp4;
do name=`echo $i | cut -d'.' -f1`;
echo $name;

In this code block the i value picks up the file names ending with .mp4
Then the name block uses . as the delimiter and picks up the first column i.e. if you file name is A_B.mp4 it will show A_B in the value of $name.
There's one limitation in reading the file name from the left. If you file name has dots in between e.g. file name is A.B.mp4 it will pick up A only.

And anywhere you invoke $name variable it should be enclosed in "" i.e. "$name"
 
Last edited:
Have you thought about using a Windows software like DVDfab which most probably uses FFMPEG as backend. For close to lossless quality, you set C=17,but that ends up keeping the size same or sometimes even increasing it. I settled on C=20 and later on shifted to just selecting a bit-rate of my choice which helped me reduce the size of the h265 files by 40%. If you have read the documentation, they say that C=23 is the default value for h265 which would give the same quality as default settings for h264. But since you are encoding a H264 file, I would say, just go with C=20 or a bit-rate of your choice and you should get a file with similar quality.
 
Marcus Fenix said:
Can you paste the names of a few files here with the unicode samples?

And BTW if your file names have space in between always enclose the file name variable in ".

for i in *.mp4;
do name=`echo $i | cut -d'.' -f1`;
echo $name;

In this code block the i value picks up the file names ending with .mp4
Then the name block uses . as the delimiter and picks up the first column i.e. if you file name is A_B.mp4 it will show A_B in the value of $name.
There's one limitation in reading the file name from the left. If you file name has dots in between e.g. file name is A.B.mp4 it will pick up A only.

And anywhere you invoke $name variable it should be enclosed in "" i.e. "$name"
Click to expand...
As you said I am providing the conetnts of the file which consists of non ASCII characters below

గోదా దేవి చరిత్ర _ Unknown old history of goda devi _ Tirumala _ United original.mp4
గోవిందరాజస్వామి vlog _ Tirumala sri govindaraja swamy temple _ dhasaavathaaram history ( 240 X 426 ).mp4

I am providing the script to run in Linux terminal emulator with some changes below

for i in *.mp4;
do name=`echo $i | cut -d'.' -f1`;
echo $name;
ffmpeg -y -i "$i" -c:v libx265 -x265-params pass=1 -an -f mp4 null /dev/null && \
ffmpeg -i "$i" -c:v libx265 -x265-params pass=2 -c:a aac -b:a 48k -ar 44100 "${name}_compressed.mp4";
done


Can you try to specify any errors crept in and provide a correct script which runs in proper way.
 
I had to remove the non-ASCII values from the new name.

for i in *.mp4
do name=`echo $i | cut -d'.' -f1`
non_ascii_name="${name//[^[:ascii:]]/}"
echo "$non_ascii_name"
ffmpeg -y -i "$i" -c:v libx265 -x265-params pass=1 -an -f mp4 null /dev/null && \
ffmpeg -i "$i" -c:v libx265 -x265-params pass=2 -c:a aac -b:a 48k -ar 44100 "${non_ascii_name}_compressed.mp4";
done

Please note that I didn't run ffmpeg as I don't have it installed.

Just for my curiosity how many files do you have that you want to convert?

I use Handbrake to convert multiple files from a single folder.
You scan the folder in one go and add all the files to the queue.

If you have Intel QuickSync or nVidia GPU its much faster than CPU encoding.

nVidia CUDA gives a good balance of quality and speed in my opinion. Intel QuickSync loses a bit on the picture quality. My RTX 2080 can encode a 1080p file of 1 hour duration to HEVC 480p in around 6-7 mins.

Now bitrate is the thing that has the biggest impact on the file size. I usually go with 1024 Kbps for files with resolution of above 854x480 for HEVC.

If you want I can share my HEVC config json which you can import into Handbrake and tweak as per your needs.

1638798016734.png

1638798121510.png


1638798145045.png


1638798233465.png
 
For optimum quality use crf method, in handbrake
High quality stuff with pretty compact size.

If only these software corps handle the audio better.
 
JMak said:
For optimum quality use crf method, in handbrake
High quality stuff with pretty compact size.

If only these software corps handle the audio better.
Click to expand...
I used the CRF setting initially (I think it was 18) but then switched to manual bitrate control with 23.976 fps as it was good enough quality for p**n from my perspective. :p
 
Last edited:
Marcus Fenix said:
I used the CRF seeting initially (I think it was 18) but then switched to manual bitrate control with 23.976 fps as it was good enough quality for p**n from my perspective. :P
Click to expand...
Haaa haaa.. Then no worries [emoji6][emoji6][emoji6]
But for hevc 480p you can use 25 or even 26 easily, provided the source is hd or fhd
 
Ok do you mean that handbrake is better than ffmpeg.

There are other tools like x media recode, avidemux etc. Which is best among these

I think that the file converted using command line tool has high quality than the ones converted by guis.
 
rupeshforu3 said:
Ok do you mean that handbrake is better than ffmpeg.

There are other tools like x media recode, avidemux etc. Which is best among these

I think that the file converted using command line tool has high quality than the ones converted by guis.
Click to expand...
Indeed handbrake has come a long way.
Just initial setup and everything else is automated.
 
deusExMachina said:
doesn't handbrake use ffmpeg internally?
also @JMak what CRF do you use with HEVC for SD videos? like when ripping DVDs
Click to expand...
I think all encoding softwares use ffmpeg.
For sd, you should stick to 25.
Telling you by seeing the content that I have diwbloaded and compared
 
I have solved this issue by the following option

-cpucount 3

Where my processor consists of 4 cores.

Another issue is suppose the input mp4 video file consists of bitrate 160 kbps then after converting this particular file the output mp4 video file consists of bitrate 250 kbps.

In the above bitrate is video bitrate not audio bitrate.

I have seen the properties of source and output mp4 video files in media info.

Can you suggest how to convert the x264 mp4 video file into x265 mp4 video file with the same video bitrate as input video file.
 
Hi ffprobe tool can be used to extract the bitrate of the source input mp4 video file which later can be used in ffmpeg command as below.

ffprobe -v error -select_streams v:0 -show_entries stream=bit_rate -of default=noprint_wrappers=1 input.mp4

Actually the above process is working with one file I mean I have used ffprobe tool to get the video bitrate and after that I have used it by invoking ffmpeg command by providing the bitrate value.

I want to apply the same process in a Linux shell script and run it and so I have developed a small shell script as below.

for i in *.mp4;

do name=`echo $i | cut -d'.' -f1`;
echo $name;

$temp=`ffprobe -v error -select_streams v:0 -show_entries stream=bit_rate -of default=noprint_wrappers=1 $i`;

$br=`echo $temp | cut -d'=' -f1`;

ffmpeg -y -i "$i" -cpucount 3 -c:v libx265 -b:v $br -preset medium -c:a libfdk_aac -b:a 52k -ar 44100 "${name}_compressed.mp4";

# echo $br;
# echo $temp;
done

Upon running the above script I am getting the following errors.

syntax error near unexpected token `$temp=`ffprobe -v error -select_streams v:0 -show_entries stream=bit_rate -of default=noprint_wrappers=1 $i`'

./ffmpeg_Linux_script_with variable.sh: line 4: =: command not found

./ffmpeg_Linux_script_with variable.sh: line 5: =: command not found

[NULL @ 0x30ab980] Unable to find a suitable output format for 'medium'

Can you suggest what's wrong with the above script.
 
temp=`ffprobe -v error -select_streams v:0 -show_entries stream=bit_rate -of default=noprint_wrappers=1 $i`;

This is what it should be.

When you are assigning variables it doesn't have $ at front e.g.
var=1

When you invoke the variable value then you put $ at front e.g.
echo $var


In temp you are getting this output-
bit_rate=1020081

Please note that its in Bits/sec and not Kbps which is what I prefer.

Now br=`echo $temp | cut -d'=' -f1`;

Here you are taking = as the delimiter and f1 gets you the first column i.e. the string "bit_rate". You need the 2nd column.

It should be br=`echo $temp | cut -d'=' -f2`;

Now I am not sure if ffmpeg takes bps as the input or it prefers Kbps.

I use this for the Overall bitrate-
RAW_FILE_BITRATE=`mediainfo "$i"|awk '$0~/Overall bit rate/{gsub(/[^0-9,.]/,"");printf("%s",$0)}'`;
 
Back
Top