Insulated concrete sandwich panels consist of two layers of concrete wythe separated by a foam insulation. The objective of this study is to develop an innovative fiber reinforced polymer (FRP) shear plate connector with specially designed anchoring schemes, and study its effects on the flexural behavior of insulated concrete sandwich panels, in terms of stiffness, strength, and applicability for roof/floor constructions, based on combined experimental investigation and Finite Element (FE) analysis. Three groups of 2743×610×254mm3 (9'×2'×10″) concrete panels were constructed with continuous, segmental, and discrete FRP shear plate connectors, with two panels for each group. Additionally, two solid concrete panels were constructed as baselines. These specimens were tested under bending until failure. FE analysis was conducted on the panels. The accuracy of the FE model was proven through good correlations between test and FE results for two panels with continuous shear connectors, one solid panel and one panel with segmental shear connector. It can be concluded that the FRP plate can be used to transfer shear between the two concrete wythes, achieving a composite panel, which can meet ACI requirements for roof/floor applications. Different types of shear connectors, representing different degrees of composite action, affect both the strength and stiffness of the panels. This degree of composite action can be predicted with the FE model developed in this study. Continuous and segmental connectors perform much better than discrete connectors. The use of discrete shear connectors is not recommended. Unlike other proprietary FRP shear connectors, the FRP plate shear connectors developed in this study can be cut from commercially available FRP plates and are expected to be widely used for insulated concrete sandwich panels.