When confronted with toddlerhood let's face it, most parents just give up and resort to the nursery "until he reaches a stage where he can behave in church". I can sympathise, but that's a lot like deciding not to talk to your child until he can use full sentences, or not to bother with giving him dishes until he figures out table manners. The biggest thing your toddler is learning over these two years is to relate: to relate the ME to the NOT-ME, to relate his body to the space he occupies, to relate physical objects to his control of them, and to relate how living things respond to his own responses. Will he learn to relate to God? A lot depends on whether we let him respond to God during his toddler years.  

So, hard as it is, bring your toddler to church. For my part, I'll provide the missing Good Advice: Depending on either your instincts or your analysis at the time,

Let your child wander within limits. A person who has just learned how to walk has a highly developed spacial awareness. Such a person would be as frustrated by not being allowed to move around, as you would be if you were told to keep your eyes closed.

• Limit your child to what you are comfortable with. You may be comfortable with your child is anywhere on the Nave (or "sanctuary") floor, but not if she goes into the Chancel or climbs on the pews. If your anxiety level peaks don't worry about being overly controlling: rein her in. Then remember, and enforce that limit from then on. As soon as she breaches the limit, fetch her back a distance, state the limit, and let her go again.
• Don't reward fusses. If you have to remove your child, take him into the narthex or other area, but give him LESS freedom than he'd have in church. For example, hold him still on your lap. If he's fussing from tiredness or overstimulation that should calm him better than letting him run around, and if not it will make being in church preferable to being removed. Eventually - over four to six months of this treatment - he may start to quiet as soon as you stand up with him.
• Beware of growth spurts which cause ravenous hunger and nervous stress. Have biscuits or cereal for your toddler to snack on, and something cuddly for her to curl up with if she's tired.
• DON'T try to DISTRACT a fussy toddler -- DO try to ENGAGE her. A liturgical church is full of vivid sensual experiences. Draw his attention to the action of the Eucharist - "Look, Father has the Holy Bread! Do you see the Holy Bread? Where's the Holy Bread?" - or to the stained glass, or to familiar symbols like the cross, or to other worshippers. After a baptism, let him touch the water, and draw a cross on his forehead like the one on the baby's forehead. DEFINATELY let him take the communion. Provide toys if you can, that also draw the focus back to the worship: bible-based activity blankets or stuffed bible-toys, or scriptural soft books. But remember the first rule of toddlerhood: there ARE no quiet toys. You just have to modulate the noise the best you can, and try not to worry.
• DON'T stifle your child's response to God. If he points and shouts "Cross! Cross!" at the Gospel procession, great! You can reinforce his enthusiasm AND model appropriate vocal control at the same time, by murmuring "Yes, my dear, that's the Cross! Good for you". But vocal modulation is generally not a developmentally appropriate emphasis at this age.
• DO bow your head and steeple your hands, make the sign of the cross, bow at the name of Jesus, and teach your child to do these things too. Children like games with rules, and these are the "rules" for worshipping. Later when they're teenagers they can rebel by not bowing their heads, and they won't have to run away to feel appropriately rebellious.
• Give your child money to put in the offering plate as soon as she recognises what money is and can be counted on not to swallow it. Make the offering the purpose of your child's allowance: she gets two pennies SO that she can give one to God. As soon as she understands counting to 10, give her 10 pennies, and insist that one is for God. As she gets older, give her her allowance in multiples of 10 right before church (or on Saturday night if you feel giving allowance constitutes breaking the Sabbath) so that it's always easy to tithe. Inure yourself to the sound of pennies falling on the floor: if enough supporters had learned to tithe as children, the church could afford a carpet.