Trying to make new choices about video codec (and maybe container)

[TCmullet]

Please forgive this probably being old hat to many.  Esp. seeing as I've been working w/video encoding myself for many years, starting back in the (simpler) Mpeg2 days.  I've been using Avidemux for several years, mostly for gop-editing H.264 mp4 files, as well as .TS.  But I have run the X264 a few times.  (A very few times.)

H.264 is what I am generally given whether from my h.264 capture devices or from an external source.  I've hit a crossroads in what I'm doing and conclude that if possible, I need to stop using another encoder to "top quality" encode to h264/mp4 and be startlingly open to consider a different target.  H264 was necessary for me as all video I have worked with needed to be uploadable to Youtube.  But now I realize I have a need NOT for Youtube compatibility, but rather for SMALLNESS, as well as full playability on anyone's PC or Mac, etc.

And I am hoping the answer lies in using Avidemux to encode.  However, when I look at the encoders "in it", there's only several, including X264 (which is an implemenation of H.264) and X265.  I've heard 265 is much more efficient.  And I assume that the target CONTAINER would still be MP4.

I'd greatly appreciate advice on where I should go.

If the answer is X265, then I would then ask how to go about dealing with the fact that (just as with X264) there are way more parameters for me to "mess up" than I'd like.  I really want something simple that will get a file as small as possible and I am willing to do SOME preliminary (short) test runs to if it's a matter of deciding what "Q" or something to settle on for the videos I'm working with.  (I cannot do experimentation with each video.  I have to learn what to use and what settings to use, then I'll use those for scads of videos.)

Update:  Oops, apparently YT has expanded what they accept to INCLUDE h265.  But this really doesn't affect my question as I'm no longer that interested in making videos for YT uploading.  Here's the YT rules:
https://support.google.com/youtube/troubleshooter/2888402?hl=en

[eumagga0x2a]

The general rule is to avoid re-encoding whenever possible. When the source video is already H.264 and doesn't need to be filtered, leave it as-is and just cut it at GOP boundaries.

Nothing is wrong about continuing to use the MP4 container as long as you don't try to mux streams compressed with codecs MP4 doesn't support, which means that you should use MKV then.

now I realize I have a need NOT for Youtube compatibility, but rather for SMALLNESS, as well as full playability on anyone's PC or Mac, etc.

Both requests a mutually exclusive. For best compression rate, re-encode to HEVC using x265 (increasing GOP length improves compression further at the cost of seek granularity). For compatibility with anyone's PC or Mac (the target must be able to decode video in hardware, the computational cost of HEVC decoding in software is prohibitive) stick with H.264, accepting bigger file sizes.

You can trade seek granularity for better compression with H.264 as well, this is a question of personal priorities.

It may also happen that the new AV1 codec will supercede HEVC before the latter gains wide device support. Who knows.