Abstract: Panoramic symbol spotting(PSS) of power design CAD drawings is of great significance for power system intelligence. Existing PSS methods, when applied to power scenarios, encounter difficulties in handling power design CAD drawings with primitives of a large amount and uneven sizes, and cannot thoroughly utilize the text annotation information in the drawings. To address this, PFL-Net is proposed to construct an encoder based on locally sensitive reordering and a cross-covariance self-attention mechanism, which reduces the attention complexity to adapt to a long sequence of primitives. The model also applies a factorized multi-head bilinear pooling method to fuse the text information into vector graphic primitives. PFL-Net surpasses existing PSS methods by 9.97 percent in panoramic quality under the experiment using power design datasets, and improves prediction speed from second level to sub-second level, indicating its greater applicability.