One might say that hockey skills are hockey skills and whatever helps you score at even strength should carry over. Thinking more about it, though, there are lot of aspects of the game that are in one but not the other. If your team excels at the breakout, On the Forecheck or has a mean neutral-zone trap then that won't help you too much when the other team simply clears the puck and waits for you at the blue line.
In the analytical community, the most-cited metric for power-play quality is shooting rate. See the recent article that got me thinking about this by Derek Zona or the definitive article on the subject by Gabe Desjardins. Since I am using 5-on-4 shots for per 60, the most obvious stat for comparison is 5-on-5 shots for per 60. So the question at hand is how correlated are 5-on-5 and 5-on-4 shooting rates.
Using data from BTN, over the last four full seasons the correlation between shooting rates 5-on-5 and 5-on-4 is 0.4. Normally I would put out an R^2 interpretation, but for this relationship it isn't one driving the other but some hockey skills and tactics driving both. Here is a graph showing the relationship:
With a correlation of 0.4 one could say there is a relationship between the two, but it isn't particularly strong. Luck is obviously one factor. We tend to focus on shooting rates because that reduces the luck factor, but it is still present. That is particularly true for the power play due to reduced sample sizes.
Here is a chart showing the average difference between 5-on-4 shooting rate and the one predicted by the above regression formula for each team. You can think of this as how good they have been 5-on-4 compared to offensively 5-on-5.
As you can see, the Sharks, Ducks and Red Wings are outliers at the positive end. Their power plays have performed better than you would expect based on their 5-on-5 shooting rates. The Sharks under McLellan have been very strong with the man advantage. The huge outlier in the graph above is the ungodly 72.6 5-on-4 shots/60 they put up last year. They currently lead the league this year at 68.8. Getting slightly off-topic, I think a safe prediction is for the Sharks to improve their power-play results since they top the league in 5-on-4 SF/60 but are only 10th in PP%.
On the bad end, the Edmonton Oilers stand alone. This is saying something, since the Oilers haven't exactly been machine gunning pucks at the net 5-on-5. Going by 5-on-4 shots for per 60 they have finished the last four years 30th, 30th, 30th and, you guessed it, 30th. This year Ryan Nugent-Hopkins can barely miss on the power play and perhaps he's helped their shooting rate out as well since they are currently all the way up to 24th in the league.
Given how consistently the Sharks, Ducks and Red Wings have outperformed expectations with the man advantage and how terrible the Oilers' power play has been, I think it's pretty clear that there is some skill component of the power play that is distinct from 5-on-5.