Nobody said there were rules like that. It's about encouraging people to participate.
You're asking people to make an engine, you've not specified what you have in mind, you're talking about a very large topic. Are you looking at people developing a game engine with shaders, physics, scripting and an editor? Or are you looking at a simple text adventure engine with scripting? What I'm talking about is the difference between writing a novel and a short story.
If the former is more in your mind, you will need something a little more serious to encourage people. Serious means people we know who can judge fairly, therefore, somebody who's qualified to do it. I mean the Karate competitions I entered weren't judged by 6th Kyus on the grounds that they 'know Karate', or the Storytelling competition wasn't judged by the local drunkard because he could tell stories.
The latter end of the scale is probably not all that exciting to show off and is probably less challenging. Perhaps give us a better idea.
If you're just having a comp just for 'fun' you might want something a little more small scale? ESPECIALLY if you're expecting it to be open source.