I'm not wanting to appear condescending but far and away the most likely cause of loss of control for someone flying an RC model for the first time is 'pilot error' caused by inexperience. You would be hard pushed to find anyone who has ever tried to fly an RC model on their own, unassisted, who has not lost control and crashed, so it's certainly not just you.
Sure the model may not be properly trimmed out, that's quite normal for a maiden flight and that's why they put 'trims' on transmitters. An experienced flier would almost certainly be able to deal with it without any issues at all.
Best advise would be to go find a club in your area or at least an experienced flying buddy to help you set the plane up and to get you over the first few flights.
As for getting the CG right; are you using the recommended battery?.. How many mm back was your CG position? Within reason a CG that is forward of the recommended spot (i.e. nose heavy) should not cause any serious issues. On it's own that it wont make the plane loop.