Q:

Does Bitcoin protocol define how miners do the selection from their set of txns?

We know that miners select a set of txs from their local memory pool and put them into a block before they do PoW. 
I wonder whether this selection process is defined by the protocol or do even "honest" miners can select them in the way they want, e.g. they select the subset that maximizes tx fee.

A:

No, the selection process of transactions to include in a block is not defined in the protocol. The only transaction that miners have to include in a block is the coinbase transaction, i.e the transaction that reward them with new bitcoin(s) if they successfully mine the block. But even if miners don't have to include other transactions in a block, they will because it allow them to collect the transaction fees.  
Miners in general choose to include transactions with the higher fees. In the bitcoin core client, the transaction date also plays a role: older transactions are prioritized over newer ones, so that transaction with a lower transaction fees still get a chance to be included in a block, but later.
Edit: as per Pieter Wuille comment, older transactions arent prioritized anymore from v0.15.0

