Mix infused olive oil, salt, and pepper in small bowl. Place swordfish steaks in small zip lock baggies with a good amount of marinade covering entire portion and refrigerate. Allow fish to marinate for 30 minutes or so prior to cooking.
Preheat well oiled grill to medium high heat. Remove swordfish from marinade shaking to remove excess oil. Grill swordfish for 3-4 minutes on each side or until grill marks begin to appear. Serve with your favorite grilled veggies that can also be marinated with infused oils.